LINQ, Unable to create a constant value of type XXX. Only primitive types or enumeration types are supported in this context

You can't use Contains with non-primitive values. Do

Where(l => l.Courses.Select(c => c.CourseId).Contains(courseId)

(or the Id field you use).

If you are using a DbContext, you can query the .Local collection, and the == operator will work also with objects:

public void RemoveCourse(int courseId)
{
    using (var db = new AcademicTimetableDbContext())
    {
        var courseFromDb = db.Courses.Find(courseId);

        db.Lecturers.Load() //this is optional, it may take some time in the first load

        //Add .Local to this line
        var toRemove = db.Lecturers.Local 
                        .Where(l => l.Courses.Contains(courseFromDb)).ToList();

        foreach (var lecturer in toRemove)
        {
            lecturer.Courses.Remove(courseFromDb);
        }

        db.SaveChanges();
    }
}

The .Local is an ObservableCollection, so you can compare anything you like inside it (not limited to SQL queries which don't support object comparison). Just to make sure you get all your objects in the .Local collection you can call the db.Lecturers.Load() method before calling .Local, which brings all database entries into the Local collection.