Wordpress - Why is per_page not working with categories in WP API?

Pretty much all the URLs you are using are invalid in some way:

http://foobar.com/wp-json/wp/v2/posts/categories_name=hello

categories_name is not a valid argument for listing posts, and even if it was you are missing the ? part of the query string.

http://foobar.com/wp-json/wp/v2/posts/categories_name=hello?per_page=‌​1

This one is also missing the ?. Query parameters on a URL, which the API uses for its arguments, need to start with ?, with & for additional parameters. A correctly formatted query string looks like this:

http://domain.com/path/?argument=value&argument2=value2

So this one:

http://foobar.com/wp-json/wp/v2/posts/categories_name=hello&per_page=‌​1

Is also missing the ? but you've used & correctly this time (though are still using the invalid categories_name argument).

This one:

http://foobar.com/wp-json/wp/v2/posts/?per_page=‌​1$categories_name=hello

Is using a $ for some reason. That's not a valid way to separate parameters in a query string (and still using the invalid categories_name argument).

This one is correct:

http://foobar.com/wp-json/wp/v2/posts/?categories=4&per_page=‌​1

But based on your comment:

I thought I might be able to use -1 similar to the development of a theme but I get an error.

It sounds like you actually tried:

http://foobar.com/wp-json/wp/v2/posts/?categories=4&per_page=‌​-1

Which won't work, because -1 is an invalid value. You can only retrieve between 1 and 100 results with the API.