Why/How SOAP is Stateful?
First off, SOAP is protocol. In other way we have services like REST, WSDL, WCF and many more. SOAP can be used as protocol in all kind of services. This mean, soap provide mechanism to pass data between client and server.
By design and main purpose SOAP is data protocl to be use between server and clients. Server(service) save, preserve and response data. Soap as data can be used in both variants in service - stateful or stateless.
By default soap service is stateless. For example WSDL soap service is stateless. Simple WebAPI, REST. They are all stateless.
WCF - Single Instance is stateful. WCF - Per call is stateless.
Being stateful is not one of SOAP protocol principles or ideas behind it. Service can be stateful or stateless.
Basically Stateful means that server stores information about the client and uses that information over a series of requests. So performing one request is dependant upon the state of some other request (e.g. previous). Implementing this is possible with http protocols.
So you can have stateful or stateless SOAP - it's only a matter of how you design it.
Also please note that comparing SOAP and REST is not really correct. The first one is basically Protocol (or at least it's trying to be) and REST is just a architecture pattern/style.
I know this is not exactly answering your question but please take a look at this link: SOAP vs REST (differences) It's extremely well written and can help you understand those technologies a bit better.