Private Final vs Final Private

private final and final private has no difference. For access modifiers the order does not matter. So both of them will have similar behavior.

Try this:

public class Test {

    private final String privateFinal;
    final private String finalPrivate;

    public Test() {
        privateFinal = "Private Final";
        finalPrivate = "Final Private";
    }
}

The above code compiles absolutely fine.

As an example, here are a couple cases of things that are ok and not ok with final. There is no difference between private final and final private. To add to what @Sagar said, you can initialize the variables inline, or in the constructor, but not both.

public class TestClass {

    private final String i1;
    final private String i2;
    private final String i3 = "test"; // ok
    private final String i4; // not ok, never initialized

    TestClass() {
        i1 = "test1"; // ok
        i2 = "test2"; // ok
        i3 = "test3"; // not ok, overrides already set value
    }

    void mod() {
        i1 = "test0"; // not ok, can't edit final i1
    }
}

The Java Language Specification, section 8.3.1. Field Modifiers, says:

FieldModifier:
  (one of)
  Annotation public protected private
  static final transient volatile

If two or more (distinct) field modifiers appear in a field declaration, it is customary, though not required, that they appear in the order consistent with that shown above in the production for FieldModifier.

Which means that private final is the preferred style, but it's exactly the same as final private.

Later on, the code asked me to initialize the variable in a constructor, which obviously fails against a private final variable as it is immutable.

Members marked as final can be initialized in constructor. You can either create a prameterized constructor which accepts values from outside of class or simply initialize those members directly with declaration. If you choose later and the data type is primitive, then mark those members as static since they are going to remain same for all instances of class.

HOWEVER, it did not fail when I changed the variable to a final private... which made me interested

This won't be possible since private final and final private and are virtually the same thing and won't make any difference. However former is considered as good practice. You can follow this SO for more details regarding ordering.