Record and replay Javascript

I believe you are looking for Mutation Events.

http://www.w3.org/TR/2000/REC-DOM-Level-2-Events-20001113/events.html#Events-eventgroupings-mutationevents

Here are some resources for you:

http://tobiasz123.wordpress.com/2009/01/19/utilizing-mutation-events-for-automatic-and-persistent-event-attaching/

http://forum.jquery.com/topic/mutation-events-12-1-2010

https://github.com/jollytoad/jquery.mutation-events

Update:

In Response to comment, a very, very basic implementation:

//callback function
function onNodeInserted(){alert('inserted')}
//add listener to dom(in this case the body tag)
document.body.addEventListener ('DOMNodeInserted', onNodeInserted, false); 
//Add element to dom 
$('<div>test</div>').appendTo('body')

Like WTK said, you are getting yourself into complex territory.

I became curious by this question and implemented a proof of concept here

https://codesandbox.io/s/jquery-playground-y46pv?fontsize=14&hidenavigation=1&theme=dark

Using the demo

• Try pressing record, clicking around, press record again and then click play.
• Play creates an <iframe>, injects the original HTML and replays the user events.
• To change zoom change the REPLAY_SCALE variable in the source code.
• To change the playback speed change the SPEED variable in the source code.
• NB I only tested the jsbin on chrome 56.

Implementation details:

• It supports mousemove, click and typing. I omitted others (scroll, window resizing, hover, focus etc etc) but should be easily extensible.
• The event listeners bypass any event.stopPropagation() by using capture when listening for events on the document.
• Displaying playback in a different resolution is done using zoom CSS3.
• A transparent canvas could be overlaid to draw the trace lines of the mouse. I use just a simple div so no trace lines.

Considerations:

• Imagining we are capturing user events on a real website. Since the page served could change between now and the playback we can't rely on the client's server when replaying the recording in the iframe. Instead we have to snapshot the html, all ajax requests and resource requests made during the recording. In the demo I only snapshot the HTML for simplicity. However in practice, all extra requests would have to be stored on the server in realtime as they are downloaded on the client page. Furthermore, during playback it is important that the requests are played back with the same timeline that they were perceived by the user. To simulate the request timeline, the offset and duration of each request must also be stored. Uploading all page requests as they are downloaded on the client will slow down the client page. One way to optimize this uploading could be to md5 hash the contents of the request before they are uploaded, if the md5 hash is already present on the server, the request data need not be reuploaded. Furthermore, the session of one user can leverage the request data uploaded by another user using this hashing method.

• Careful consideration will be needed when uploading all the events. Since lots of events will be generated, this means lots of data. Perhaps some compression of the events could be made e.g. losing some of the less important mousemove events. An upload request should not be made per event to minimize number of requests. The events should be buffered until a buffer size or timeout is reached before each batch of events is uploaded. A timeout should be used as the user could close the page at any point thus losing some events.

• During playback outgoing POST requests should be mocked to prevent duplicating events elsewhere.

• During playback the user agent should be spoofed but this may be unreliable in rendering the original display.

• The custom recording code could conflict with client code. e.g. jquery. Namespacing will be required to avoid this.

• There might be some edge cases where typing and clicking may not reproduce the same resulting HTML as seen in the original e.g. random numbers, date times. Mutation observers may be required to observe HTML changes, although not supported in all browsers. Screenshots could come in useful here but might prove OTT.

Record

Save the initial DOM of the page, remove the scripts from it and also you need to change all relative URLs to absolute ones.

Then, record DOM mutations and Keyboard/Mouse event.

Replay

Start with initial saved DOM, apply mutations and events using timestamp order.

In fact, clicks will not do anything because we have removed any scripts. but because we have saved the DOM changes we can replay the effect after the click.

Replaying user actions with just Javascript is a complex problem.

First of all, you can't move mouse cursor, you can't emulate mouseover/hovers also. So there goes away a big part of user interactions with a page.

Second of all, actions, once recorded, for most of the time they have to be replayed in different environment than they were recorded in the first place. I mean you can replay the actions on screen with smaller resolutions, different client browser, different content served based on replaying browser cookies etc.

If you take a time to study available services that enable you to record website visitors actions ( http://clicktale.com, http://userfly.com/ to name a few), you'll see that none of them are capable of fully replaying users actions, especially when it comes to mouseovers, ajax, complex JS widgets.

As to your question for detecting changes made to the DOM - as Chris Biscardi stated in his answer, there are mutation events, that are designed for that. However, keep in mind, that they are not implemented in every browser. Namely, the IE doesn't support them (they will be supported as of IE 9, according to this blog entry on msdn http://blogs.msdn.com/b/ie/archive/2010/03/26/dom-level-3-events-support-in-ie9.aspx).

Relying on those events may be suitable for you, depending on use case.

As to "better more simple way to store all events". There are other ways (syntax wise), of listening for events of your choice, however handling (= storing) them can't be handled in simple way, unless you want to serialize whole event objects which wouldn't be a good idea, if you were to send information about those event to some server to store them. You have to be aware of the fact, that there are massive amount of events popping around while using website, hence massive amount of potential data to be stored/send to the server.

I hope I made myself clear and you find some of those information useful. I myself have been involved in project that aimed to do what you're trying to achive, so I know how complicated can it get once you start digging into the subject.