Why do assignment statements return a value?

Haven’t you provided the answer? It’s to enable exactly the kinds of constructs you have mentioned.

A common case where this property of the assignment operator is used is reading lines from a file...

string line;
while ((line = streamReader.ReadLine()) != null)
    // ...

My favorite use of assignment expressions is for lazily initialized properties.

private string _name;
public string Name
{
    get { return _name ?? (_name = ExpensiveNameGeneratorMethod()); }
}

To my understanding, assignment s = "Hello"; should only cause "Hello" to be assigned to s, but the operation shouldn’t return any value.

Your understanding is 100% incorrect. Can you explain why you believe this false thing?

What is the reasoning behind allowing assignment statements to return a value?

First off, assignment statements do not produce a value. Assignment expressions produce a value. An assignment expression is a legal statement; there are only a handful of expressions which are legal statements in C#: awaits of an expression, instance construction, increment, decrement, invocation and assignment expressions may be used where a statement is expected.

There is only one kind of expression in C# which does not produce some sort of value, namely, an invocation of something that is typed as returning void. (Or, equivalently, an await of a task with no associated result value.) Every other kind of expression produces a value or variable or reference or property access or event access, and so on.

Notice that all the expressions which are legal as statements are useful for their side effects. That's the key insight here, and I think perhaps the cause of your intuition that assignments should be statements and not expressions. Ideally, we'd have exactly one side effect per statement, and no side effects in an expression. It is a bit odd that side-effecting code can be used in an expression context at all.

The reasoning behind allowing this feature is because (1) it is frequently convenient and (2) it is idiomatic in C-like languages.

One might note that the question has been begged: why is this idiomatic in C-like languages?

Dennis Ritchie is no longer available to ask, unfortunately, but my guess is that an assignment almost always leaves behind the value that was just assigned in a register. C is a very "close to the machine" sort of language. It seems plausible and in keeping with the design of C that there be a language feature which basically means "keep on using the value that I just assigned". It is very easy to write a code generator for this feature; you just keep on using the register that stored the value that was assigned.