How can I use Jest to spy on a method call?

The key is using jests spyOn method on the object's prototype. It should be like this:

const spy = jest.spyOn(Component.prototype, 'methodName');
const wrapper = mount(<Component {...props} />);
wrapper.instance().methodName();
expect(spy).toHaveBeenCalled();

As found here e.g.: Test if function is called react and enzyme

Please note it is also best practice to clear the spied function after each test run

let spy

afterEach(() => {
  spy.mockClear()
})

https://facebook.github.io/jest/docs/en/jest-object.html#jestclearallmocks

I know its a bit late, but I came across this and would suggest that to test componentDidMount initiates the call to your nested method that your test should look something like:

Module

componentDidMount() {
  if (this.props.initOpen) {
    this.methodName();
  }
}

Test - Good

it('should call methodName during componentDidMount', () => {
    const methodNameFake = jest.spyOn(MyComponent.prototype, 'methodName');
    const wrapper = mount(<MyComponent {...props} />);
    expect(methodNameFake).toHaveBeenCalledTimes(1);
});

If you call componentDidMount then the assertion that methodName was called via componentDidMount is more valid.

Test - Bad

it('should call methodName during componentDidMount', () => {
    const spy = jest.spyOn(Component.prototype, 'methodName');
    const wrapper = mount(<Component {...props} />);
    wrapper.instance().methodName();
    expect(spy).toHaveBeenCalled();
}

By writing the test like this - you call the method and then assert that it was called. Which of course it will have given you just called it.