How secure is "Firefox Hello"?
WebRTC is a P2P technology and performs encryption end-to-end by default.
No metadata on Mozilla's servers. Per session encryption, so Mozilla cannot decrypt.
There may be some vulnerabilities with the technology (i.e. there are), but the design is supposed to address your questions on an architectural level.
At first here is a small list of all abbreviations used when you explain WebRTC, so if you do not know one you can have a look there: https://hacks.mozilla.org/2013/07/webrtc-and-the-ocean-of-acronyms/
Summary
Here a small summary. More detailed information below.
- connections are end-to-end-encrypted
- DTLS and SRTP is used for encrypting
- peer-to-peer connections are used when possible
- if not possible the traffic may be routed through a TURN server
- signalling servers are used to exchange metadata
- the hashes of the certificates used for the encryption of the connection are also exchanged, which makes it possible for the signalling server to perform a Man-in-the-middle attack on the connection
- STUN servers are used to get a list of IP addresses used by both conversation partners
- this may be misused for tracking users, but that is more a privacy-issue than a security-issue
Security
WebRTC is a complex technology, but there are some things which can be noted.
At first the reference implementation is open-source and so is the implementation in Firefox. However that's not directly about the technology as it. Additionally the connection is end-to-end encrypted. Non-encrypted modes are explicitly disallowed. For the encrypted connection DTLS (a variant of TLS for UDP) is used (p. 18, PDF) and the audio and video (media) communication is encrypted with SRTP (p. 6, PDF), with the same keys as used for the DTLS handshake (called DTLS-SRTP). For data channels the specification requires SCTP over DTLS. Javascript access to the "key material" used by the DTLS-SRTP connection is not allowed. So this is securely handled in the browser.
Since Firefox 38 Firefox only supports Forward Secrecy suites. In the specification it is recommend to prefer Forward Secrecy cipher suites over non-Forward-Secrecy ones - and all implementations have to support them (p. 19, PDF). Additionally it adds some information about the cipher suites which may be used:
All implementations MUST implement both DTLS 1.2 and DTLS 1.0, with the cipher suites TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 and TLS_DHE_RSA_WITH_AES_128_CBC_SHA and the DTLS-SRTP protection profile SRTP_AES128_CM_HMAC_SHA1_80.
Additionally it requires a "inspector interface which allows the user to determine the security characteristics of the media". However currently this does not seem to be implemented in Firefox. You can show some details about WebRTC by accessing about:webrtc, but as I see this may be used rather for debugging than for checking the security of the connection.
Routing
It is always tried to make a peer-to-peer connection. To do this STUN servers are used which return the public IP of the user connecting to them.
This is used for ICE which - easily said - tries multiple ways to get around the firewall and NATs to create a direct connection to the communication partner.
Image licensed under CC-BY-SA by Mozilla Hacks
If this does not work the process falls back to use TURN which proxies the traffic through a server.
Man-in-the-middle
Basically the certificates used for the encryption are exchanged via signaling channel and therefore a rogue signaling server can MITM the connection. However when so-called Identity Providers are used to verify the identity of the users connection to each other the users can be sure no MITM attack was done on the server. Firefox Hello provides a feature to use Firefox Accounts for logging in, but as I see currently no identity verification is done, so currently you have to trust the signaling servers used by Mozilla.
The connection to the signalling servers is secured as a normal HTTPS connection (although this is not required for WebRTC in general).
More information: How can MITM be performed in WebRTC?
Privacy
The issue mentioned by @schroeder is not a security but a privacy issue. Additionally it is a kind of unwanted, but unavoidable effect which leaks the local IP addresses of the user. In bad configurations in conjunction with a VPN it may even leak the real external IP. However it is discussable whether this is privacy impact.
And if you do not want to use it you can do so. In Firefox 42 features allowing add-ons to interfere and disable parts of WebRTC connections were implemented. Already before the release of Firefox 42 you could (and still can) disable it via about:config. Additionally there are addons for disabling WebRTC.
More resources: Security Considerations by the IETF in the current working draft
Want to drive into the details? Here you can find more information: https://webrtc-security.github.io/