Startup bean not called

Try the following set of annotations:

@Singleton
@Startup
public class Startup {
    @EJB
    private ProviderEJB providerEJB;

    @PostConstruct
    public void onStartup() {
        System.err.println("Initialization success.");
    }
}

You will find more details here and in this book (chapter 2).

The Startup annotation is for usage with Singleton beans, not with stateless beans. See the javadoc.

Also, @LocalBean is not needed in this case. This declares that you want an additional no-interface view, but this is only needed if the bean implements a remote or local business interface. If you omit it you get a no-interface view by default.