Original purpose of <input type="hidden">?

In short, the original purpose was to make a field which will be submitted with form's submit. Sometimes, there were need to store some information in hidden field(for example, id of user) and submit it with form's submit.

From HTML September 22, 1995 specification

An INPUT element with `TYPE=HIDDEN' represents a hidden field.The user does not interact with this field; instead, the VALUE attribute specifies the value of the field. The NAME and VALUE attributes are required.

The values of form elements including type='hidden' are submitted to the server when the form is posted. input type="hidden" values are not visible in the page. Maintaining User IDs in hidden fields, for example, is one of the many uses.

SO uses a hidden field for the upvote click.

<input value="16293741" name="postId" type="hidden">

Using this value, the server-side script can store the upvote.

I can only imagine of sending a value from the server to the client which is (unchanged) sent back to maintain a kind of a state.

Precisely. In fact, it's still being used for this purpose today because HTTP as we know it today is still, at least fundamentally, a stateless protocol.

This use case was actually first described in HTML 3.2 (I'm surprised HTML 2.0 didn't include such a description):

type=hidden
These fields should not be rendered and provide a means for servers to store state information with a form. This will be passed back to the server when the form is submitted, using the name/value pair defined by the corresponding attributes. This is a work around for the statelessness of HTTP. Another approach is to use HTTP "Cookies".

<input type=hidden name=customerid value="c2415-345-8563">

While it's worth mentioning that HTML 3.2 became a W3C Recommendation only after JavaScript's initial release, it's safe to assume that hidden fields have pretty much always served the same purpose.

I'll provide a simple Server Side Real World Example here, say if the records are looped and each record has a form with a delete button and you need to delete a specific record, so here comes the hidden field in action, else you won't get the reference of the record to be deleted in this case, it will be id

For example

<?php
    if(isset($_POST['delete_action'])) {
        mysqli_query($connection, "DELETE FROM table_name 
                                   WHERE record_id = ".$_POST['row_to_be_deleted']);
                                   //Here is where hidden field value is used
    }

    while(condition) {
?>
    <span><?php echo 'Looped Record Name'; ?>
    <form method="post">
        <input type="hidden" name="row_to_be_deleted" value="<?php echo $record_id; ?>" />
        <input type="submit" name="delete_action" />
    </form>
<?php
    }
?>