scalac compile yields "object apache is not a member of package org"
I had this issue because I had the wrong scope for my spark dependency in my pom.xml
file. This is wrong:
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>${spark.version}</version>
<scope>test</scope> <!-- will not be available during compile phase -->
</dependency>
Changing test
->
provided
will work and will not include spark in your uberjar which is what you will almost certainly want:
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>${spark.version}</version>
<scope>provided</scope>
</dependency>
You need to specify the path of libraries used when compiling your Scala code. This is usually not done manually, but using a build tool such as Maven or sbt. You can find a minimal sbt setup at http://spark.apache.org/docs/1.2.0/quick-start.html#self-contained-applications