"new" keyword in Scala

Is it when we try to instantiate java objects only?

Not at all. There is two general cases when you ommit new in scala. With singleton objects (that are oftenly used to store static functions and as a kind of factory similar to what you may seen in java):

scala> object LonelyGuy { def mood = "sad" }
defined module LonelyGuy

scala> LonelyGuy
res0: LonelyGuy.type = LonelyGuy$@3449a8

scala> LonelyGuy.mood
res4: java.lang.String = sad

With a case classes (actually, underneath there are class + object = companion pattern, e.g. having class and object with the same name):

scala> case class Foo(bar: String) 
defined class Foo


scala> Foo("baz")
res2: Foo = Foo(baz)

So when you work with a simple classes, rules are the same as with Java.

scala> class Foo(val bar: String) 
defined class Foo

scala> new Foo("baz")
res0: Foo = Foo@2ad6a0

// will be a error 
scala> Foo("baz")
<console>:8: error: not found: value Foo
       Foo("baz")

Bonus, there is a anonymous classes in scala, which can be constructed like this:

scala> new { val bar = "baz" }
res2: java.lang.Object{val bar: java.lang.String} = $anon$1@10ee5b8

scala> res2.bar
res3: java.lang.String = baz

Use the new keyword when you want to refer to a class's own constructor:

class Foo { }

val f = new Foo

Omit new if you are referring to the companion object's apply method:

class Foo { }
object Foo {
    def apply() = new Foo
}

// Both of these are legal
val f = Foo()
val f2 = new Foo

If you've made a case class:

case class Foo()

Scala secretly creates a companion object for you, turning it into this:

class Foo { }
object Foo {
    def apply() = new Foo
}

So you can do

f = Foo()

Lastly, keep in mind that there's no rule that says that the companion apply method has to be a proxy for the constructor:

class Foo { }
object Foo {
    def apply() = 7
}

// These do different things
> println(new Foo)
test@5c79cc94
> println(Foo())
7

And, since you mentioned Java classes: yes -- Java classes rarely have companion objects with an apply method, so you must use new and the actual class's constructor.

Is it when we try to instantiate Java objects only?

With Scala 3 (which should be released mid 2020, eight years later), based on Dotty: never.

Scala 3 will drop "new", as in this thread

Creator applications allow to use simple function call syntax to create instances of a class, even if there is no apply method implemented.

Example:

class StringBuilder(s: String) {
   def this() = this(s)
}

StringBuilder("abc")  // same as new StringBuilder("abc")
StringBuilder()       // same as new StringBuilder()

Creator applications generalize a functionality provided so far only for case classes, but the mechanism how this is achieved is slightly different.
Instead of an auto-generated apply method, we add a new possible interpretation to a function call f(args).