Queueable apex still executes in test context even if Enqueue Job is not called between Test.startTest() and Test.stopTest()

Well this is a great find. Lots of undocumented / poorly documented behavior as far as I can tell :p.

First, I verified on my end that I also am now receiving a job id in tests. I modified your unit test to verify that this happens both inside and outside of a Test.start/stop context. In both cases, we are now receiving job ids.

@isTest
public class MyQueueableApexTest {

    @isTest
    public static void queuableTestWithoutStartStop(){

        Id jobId = System.enqueueJob(new MyQueueableApex());
        System.debug('queuableTestWithoutStartStop [jobId]: ' + jobId);
        System.assertNotEquals('Completed' , [SELECT Id , Status FROM AsyncApexJob WHERE Id=:jobId][0].Status);

    }

    @isTest
    public static void queuableTestWithStartStop(){

        Test.startTest();
        Id jobId = System.enqueueJob(new MyQueueableApex());
        System.debug('queuableTestWithStartStop [jobId]: ' + jobId);
        Test.stopTest();

        System.assertEquals('Completed' , [SELECT Id , Status FROM AsyncApexJob WHERE Id=:jobId][0].Status);

    }
}

I couldn't find any release notes documenting this change. I did find that 85 people reported this as a problem on the trailblazer community, and Salesforce classified it as a "Known Issue - No Fix". So since it was considered an issue/bug, perhaps it got fixed?

Concerning the main question of your post, I believe this is intended behavior, though the phrasing of the documentation could certainly use some clarity. It sounds like the purpose of startTest and stopTest are not to control if asynchronous jobs are run during a test, but rather to control when, and under what context. From the Test class documentation:

You can also use this method with stopTest to ensure that all asynchronous calls that come after the startTest method are run before doing any assertions or testing.

The earlier document you linked has a paragraph which I think might explain why we see this behavior:

The startTest method does not refresh the context of the test: it adds a context to your test. For example, if your class makes 98 SOQL queries before it calls startTest, and the first significant statement after startTest is a DML statement, the program can now make an additional 100 queries. Once stopTest is called, however, the program goes back into the original context, and can only make 2 additional SOQL queries before reaching the limit of 100.

This is speaking of how DML limits behave inside and outside the start/stop methods, but I think it gives some insight into how async works as well. It sounds like Salesforce has something like a "context stack" that it uses to provide different limits to different pieces of code. Some managed packages for example get their own limits, and we see in the debug logs that as code enters and exits that context, the limits are added to and removed from the stack. Test.startTest/stopTest also use this framework to provide a new context when start and stop are used.

If we say that asynchronous jobs in a test context are run when that context is removed from the stack, then we'd see the following behavior:

1. Async jobs between start/stop are run when stop is called.
2. Async jobs between the beginning and end of a test method are run when the test method completes

So the long and the short of it is, async jobs will always execute within the confines of a test, but if you want to run assertions against the results of the test, then you need to wrap it with the start/stop method calls.

In my company's org, we found that a good design pattern for queueables is to always avoid enqueuing them within a test, by protecting the enqueue statement:

if (!Test.isRunningTest()) jobId = System.enqueueJob(myQueueable);

This helps to cleanly separate the (typically heavier) async processing from normal CPU/DML limits being incurred by the code that launches the queueable. Then on the queueable side, structure the logic so that it can be unit tested outside of an asynchronous context.

Hope that helps, -- Nate

I wrote a unit test that describes how all asynchronous code runs at the end of a unit test. This was in 2018, and still not fixed (and likely won't be).

The documentation is also incorrect, as far as I can tell (System.enqueueJob returning an ID in test context). Report a bug on Twitter to @salesforcedocs. They'll put something in for you.

If you do not call Test.stopTest, but you still want to kill any potential asynchronous jobs, use Database.rollback.

Savepoint sp = Database.setSavePoint();
Test.startTest();
// Do stuff here //
Database.rollback(sp);

Doing this will roll back all parts of the transaction, including calling future methods, queueable methods, scheduled methods, batchable methods, emails, post-commit platform events, etc.