How can you extend Java to introduce passing by reference?

The usual idiom I've seen for pass-by-reference in Java is to pass a single-element array, which will both preserve run-time type-safety (unlike generics which undergo erasure) and avoid the need to introduce a new class.

public static void main(String[] args) {
    String[] holder = new String[1];

    // variable optimized away as holder[0]
    holder[0] = "'previous String reference'";

    passByReference(holder);
    System.out.println(holder[0]);
}

public static void passByReference(String[] someString) {
    someString[0] = "'new String reference'";
}

To answer your question:

Where can this fail?

1. Final variables and enum constants
2. 'Special' references such as this
3. References that are returned from method calls, or constructed inline using new
4. Literals (Strings, integers, etc.)

...and possibly others. Basically, your ref keyword must only be usable if the parameter source is a non-final field or local variable. Any other source should generate a compilation error when used with ref.

An example of (1):

final String s = "final";
passByReference(ref s);  // Should not be possible

An example of (2):

passByReference(ref this);  // Definitely impossible

An example of (3):

passByReference(ref toString());  // Definitely impossible
passByReference(ref new String("foo"));  // Definitely impossible

An example of (4):

passByReference(ref "literal");  // Definitely impossible

And then there are assignment expressions, which seem to me like something of a judgement call:

String s;
passByReference(ref (s="initial"));  // Possible, but does it make sense?

It's also a little strange that your syntax requires the ref keyword for both the method definition and the method invocation. I think the method definition would be sufficient.

Your attempt to modify the language ignores the fact that this "feature" was explicitly left out to prevent well-known side-effect bugs from being able to happen in the first place. Java recommends to do what you are trying to archive by the use of data-holder classes:

public class Holder<T> {
  protected T value;

  public T getValue() {
    return value;
  }

  public void setValue(T value) {
    this.value = value;
  }
}

A thread-safe version would be the AtomicReference.

Now storing a single String in a class seems over-kill and most likely it is, however usually you have a data-holder class for several related values instead of a single String.

The big benefit of this approach is that what happens inside the method is very explicit. So even if you are programming on a Monday morning after an eventful weekend and the coffee machine just broke down, you still can tell easily what the code is doing (KISS), preventing several bugs from even happening in the first place, just because you forgot about that one feature of method foo.

If you think about what your approach can do that the data-holder version cannot, you'll soon realize that you are implementing something just because it is different, but effectively it has no real value.

Using AtomicReference class as holder object.

public static void main(String[] args) {
    String variable="old";
    AtomicReference<String> at=new AtomicReference<String>(variable);
    passByReference(at);
    variable=at.get();
    System.out.println(variable);
}

public static void passByReference(AtomicReference<String> at) {
  at.set("new");
}