What is the accepted way to send 64-bit values over JSON?
Javascript's Number type (64 bit IEEE 754) only has about 53 bits of precision.
But, if you don't need to do any addition or multiplication, then you could keep 64-bit value as 4-character strings as JavaScript uses UTF-16.
For example, 1 could be encoded as "\u0000\u0000\u0000\u0001". This has the advantage that value comparison (==, >, <) works on strings as expected. It also seems straightforward to write bit operations:
function and64(a,b) {
var r = "";
for (var i = 0; i < 4; i++)
r += String.fromCharCode(a.charCodeAt(i) & b.charCodeAt(i));
return r;
}
There is in fact a limitation at JavaScript/ECMAScript level of precision to 53-bit for integers (they are stored in the mantissa of a "double-like" 8 bytes memory buffer). So transmitting big numbers as JSON won't be unserialized as expected by the JavaScript client, which would truncate them to its 53-bit resolution.
> parseInt("10765432100123456789")
10765432100123458000
See the Number.MAX_SAFE_INTEGER
constant and Number.isSafeInteger()
function:
The
MAX_SAFE_INTEGER
constant has a value of9007199254740991
. The reasoning behind that number is that JavaScript uses double-precision floating-point format numbers as specified in IEEE 754 and can only safely represent numbers between-(2^53 - 1)
and2^53 - 1
.Safe in this context refers to the ability to represent integers exactly and to correctly compare them. For example,
Number.MAX_SAFE_INTEGER + 1 === Number.MAX_SAFE_INTEGER + 2
will evaluate totrue
, which is mathematically incorrect. SeeNumber.isSafeInteger()
for more information.
Due to the resolution of floats in JavaScript, using "64-bit floating point numbers" as you proposed would suffer from the very same restriction.
IMHO the best option is to transmit such values as text. It would be still perfectly readable JSON content, and would be easy do work with at JavaScript level.
A "pure string" representation is what OData specifies, for its Edm.Int64
or Edm.Decimal
types.
What the Twitter API does in this case, is to add a specific ".._str":
field in the JSON, as such:
{
"id": 10765432100123456789, // for JSON compliant clients
"id_str": "10765432100123456789", // for JavaScript
...
}
I like this option very much, since it would be still compatible with int64 capable clients. In practice, such duplicated content in the JSON won't hurt much, if it is deflated/gzipped at HTTP level.
Once transmitted as string, you may use libraries like strint – a JavaScript library for string-encoded integers to handle such values.
Update: Newer versions of JavaScript engines include a BigInt object class, which is able to handle more than 53-bit. In fact, it can be used for arbitrarily large integers, so a good fit for 64-bit integer values. But when serializing as JSON, the BigInt value will be serialized as a JSON string - weirdly enough, but for compatibility purposes I guess.
This seems to be less a problem with JSON and more a problem with Javascript itself. What are you planning to do with these numbers? If it's just a magic token that you need to pass back to the website later on, by all means simply use a string containing the value. If you actually have to do arithmetic on the value, you could possibly write your own Javascript routines for 64-bit arithmetic.
One way that you could represent values in Javascript (and hence JSON) would be by splitting the numbers into two 32-bit values, eg.
[ 12345678, 12345678 ]
To split a 64-bit value into two 32-bit values, do something like this:
output_values[0] = (input_value >> 32) & 0xffffffff;
output_values[1] = input_value & 0xffffffff;
Then to recombine two 32-bit values to a 64-bit value:
input_value = ((int64_t) output_values[0]) << 32) | output_values[1];
The JS number representation is a standard ieee double, so you can't represent a 64 bit integer. iirc you get maybe 48 bits of actual int precision in a double, but all JS bitops reduce to 32bit precision (that's what the spec requires. yay!) so if you really need a 64bit int in js you'll need to implement your own 64 bit int logic library.