How can I calculate the median of values in SQLite?
There is a log table with timestamp, label, and latency. We want to see the latency median value of each label, grouped by timestamp. Format all latency value to 15 char length with leading zeroes, concatenate it, and cut half positioned value(s).. there is the median.
select L, --V,
case when C % 2 = 0 then
( substr( V, ( C - 1 ) * 15 + 1, 15) * 1 + substr( V, C * 15 + 1, 15) * 1 ) / 2
else
substr( V, C * 15 + 1, 15) * 1
end as MEDST
from (
select L, group_concat(ST, "") as V, count(ST) / 2 as C
from (
select label as L,
substr( timeStamp, 1, 8) * 1 as T,
printf( '%015d',latency) as ST
from log
where label not like '%-%' and responseMessage = 'OK'
order by L, T, ST ) as XX
group by L
) as YY
There is an extension pack of various math functions for sqlite3. It includes group functions like median.
It will be more work getting this going than CL's answer, but might be worthwhile if you think you will need any of the other functions.
http://www.sqlite.org/contrib/download/extension-functions.c?get=25
(Here is the guide for how to compile and load SQLite extensions.)
From description:
Provide mathematical and string extension functions for SQL queries using the loadable extensions mechanism. Math: acos, asin, atan, atn2, atan2, acosh, asinh, atanh, difference, degrees, radians, cos, sin, tan, cot, cosh, sinh, tanh, coth, exp, log, log10, power, sign, sqrt, square, ceil, floor, pi. String: replicate, charindex, leftstr, rightstr, ltrim, rtrim, trim, replace, reverse, proper, padl, padr, padc, strfilter. Aggregate: stdev, variance, mode, median, lower_quartile, upper_quartile.
UPDATE 2015-04-12: Fixing "undefined symbol: sinh"
As has been mentioned in comments, this extension may not work properly despite a successful compile.
For example, compiling may work and on Linux you might copy the resulting .so file to /usr/local/lib
. But .load /usr/local/lib/libsqlitefunctions
from the sqlite3
shell may then generate this error:
Error: /usr/local/lib/libsqlitefunctions.so: undefined symbol: sinh
Compiling it this way seems to work:
gcc -fPIC -shared extension-functions.c -o libsqlitefunctions.so -lm
And copying the .so
file to /usr/local/lib
shows no similar error:
sqlite> .load /usr/local/lib/libsqlitefunctions
sqlite> select cos(pi()/4.0);
---> 0.707106781186548
I'm not sure why the order of options to gcc matters in this particular case, but apparently it does.
Credit for noticing this goes to Ludvick Lidicky's comment on this blog post
Let's say that the median is the element in the middle of an ordered list.
SQLite (4 or 3) does not have any built-in function for that, but it's possible to do this by hand:
SELECT x
FROM MyTable
ORDER BY x
LIMIT 1
OFFSET (SELECT COUNT(*)
FROM MyTable) / 2
When there is an even number of records, it is common to define the median as the average of the two middle records. In this case, the average can be computed like this:
SELECT AVG(x)
FROM (SELECT x
FROM MyTable
ORDER BY x
LIMIT 2
OFFSET (SELECT (COUNT(*) - 1) / 2
FROM MyTable))
Combining the odd and even cases then results in this:
SELECT AVG(x)
FROM (SELECT x
FROM MyTable
ORDER BY x
LIMIT 2 - (SELECT COUNT(*) FROM MyTable) % 2 -- odd 1, even 2
OFFSET (SELECT (COUNT(*) - 1) / 2
FROM MyTable))