Build a readability index

Perl 120 bytes

#!perl -pa0
s@\w+|([.!?])@$s+=$#-,lc($&)=~s![aeiou]+\B|([aeiouy]$)!$y+=1-$#-/3!ger@ge}
{$_=206.835-1.015*@F/$s-84.6*$y/@F

Sample I/O:

$ perl flesch-kincaid.pl < input1.dat
110.730040322581

$ perl flesch-kincaid.pl < input2.dat
65.6097727272728

$ perl flesch-kincaid.pl < input2.dat
1.71366197183096

Counting of syllables is done by assuming that each vowel cluster is a single syllable, except for lone vowels at the end of a word, which are only counted two thirds of the time; a heuristic that seems to be fairly accurate.


K&R c -- 188 196 199 229 characters

With the spec changed to specify a function I can get of lot of the c overhead out of the count. Also changing to use Strigoides' syllable counting hack which is better than my formula tweak and extended to deal with overcounting of words.

After I found a shorter way to do the vowel detection which was sadly based on stdchr, I had incentive to squeeze a few more out of the bit-twiddling abomination I've been using so that I didn't have to be boring.

d,a,v,s,t,w;float R(char*c){for(;*c;++c){s+=*c=='.';if(isalpha(*c)){
w+=!a++;d=(*c&30)>>1;if(*c&1&(d==7|((!(d&1))&(d<6|d>8)))){t+=!v++;}
else v=0;}else v=a=0;}return 206.835-1.*w/s-82.*t/w;}

The logic here is a simple state machine. It counts sentences by periods only, words by strings of alphabetic characters, and syllables as strings of vowels (including y).

I had to diddle the constants a little to get it to come out with the right figures, but I've borrowed Strigoides' trick of just undercounting the syllables by a fixed fraction.

Un-golfed, with comments and some debugging tools:

#include <stdlib.h>
#include <stdio.h>
d,a,/*last character was alphabetic */
  v,/*lastcharacter was a vowel */
  s, /* sentences counted by periods */
  t, /* syllables counted by non-consequtive vowels */
  w; /* words counted by non-letters after letters */
float R/*eadability*/(char*c){
  for(;*c;++c){
    s+=*c=='.';
    if(isalpha(*c)){ /* a letter might mark the start of a word or a
               vowel string */
      w+=!a++; /* It is only the start of a word if the last character
              wasn't a letter */
      /* Extract the four bits of the character that matter in determining
       * vowelness because a vowel might mark a syllable */
      d=(*c&30)>>1;
      if( *c&1  & ( d==7 | ( (!(d&1)) & (d<6|d>8) ) ) 
      ) { /* These bits 7 or even and not 6, 8 make for a
         vowel */
    printf("Vowel: '%c' (mangled as %d [0x%x]) counts:%d\n",*c,d,d,!v);
    t+=!v++;
      } else v=0; /* Not a vowel so set the vowel flag to zero */
    }else v=a=0; /* this input not alphabetic, so set both the
            alphabet and vowel flags to zero... */
  }
  printf("Syllables: %3i\n",t);
  printf("Words:     %3i       (t/w) = %f\n",w,(1.0*t/w));
  printf("Sentences: %3i       (w/s) = %f\n",s,(1.0*w/s));
  /* Constants tweaked here due to bad counting behavior ...
   * were:       1.015     84.6 */
  return 206.835-1.   *w/s-82. *t/w;
}
main(c){
  int i=0,n=100;
  char*buf=malloc(n);
  /* Suck in the whole input at once, using a dynamic array for staorage */
  while((c=getc(stdin))!=-1){
    if(i==n-1){ /* Leave room for the termination */
      n*=1.4;
      buf=realloc(buf,n);
      printf("Reallocated to %d\n",n);
    }
    buf[i++]=c;
    printf("%c %c\n",c,buf[i-1]);
  }
  /* Be sure the string is terminated */
  buf[i]=0;
  printf("'%s'\n",buf);
  printf("%f\n",R/*eadability*/(buf));
}

Output: (using the scaffold from the long version, but the golfed function.)

$ gcc readability_golf.c
readability_golf.c:1: warning: data definition has no type or storage class
$ ./a.out < readability1.txt 
'I would not, could not, in the rain.
Not in the dark, not on a train.
Not in a car, not in a tree.
I do not like them, Sam, you see.
Not in a house, not in a box.
Not with a mouse, not with a fox.
I will not eat them here or there.
I do not like them anywhere!
'
104.074631    
$ ./a.out < readability2.txt
'It was a bright cold day in April, and the clocks were striking thirteen.
Winston Smith, his chin nuzzled into his breast in an effort to escape
the vile wind, slipped quickly through the glass doors of Victory Mansions,
though not quickly enough to prevent a swirl of gritty dust from entering
along with him.
'
63.044090
$ ./a.out < readability3.txt 
'When in the Course of human events, it becomes necessary for one people to
dissolve the political bands which have connected them with another, and to
assume among the powers of the earth, the separate and equal station to
which the Laws of Nature and of Nature's God entitle them, a decent respect
to the opinions of mankind requires that they should declare the causes
which impel them to the separation.
'
-1.831667

Deficiencies:

  • The sentence counting logic is wrong, but I get away with it because only one of the inputs has a ! or a ?.
  • The word counting logic will treat contractions as two words.
  • The syllable counting logic will treat those same contractions as one syllable. But probably overcounts on average (for instance there is counted as two and many words ending in e will be counted one too many), so I've applied a constant factor of 96.9% correction.
  • Assumes a ASCII character set.
  • I believe the vowel detection will admit [ and {, which clearly isn't right.
  • Lots of reliance on K&R semantics makes this ugly, but hey, it's code golf.

Things to look at:

  • I am (momentarily) ahead of both python solution here, even if I am trailing the perl.

  • Get a load of the horrible thing I did for detecting vowels. It makes some sense if you write the ASCII representations out in binary and read the comment in the long version.


Python, 202 194 188 184 171 167 characters

import re
def R(i):r=re.split;w=len(r(r'[ \n]',i));s=r('\\.',i);y=r('[^aeiou](?i)+',i);return 206.835-1.015*w/(len(s)-s.count('\n'))-84.6*(len(y)-y.count(' ')-2)*.98/w

First, get the total number of words by splitting along spaces and newlines:

w=len(r(r'[ \n]',i))

Then, the formula. Sentence and syllable counts are only used once, so they are embedded in this expression.

Sentences are simply the input split along ., with newlines filtered out:

s=r('\\.',i);s=len(s)-s.count('\n')

Syllables consist of the input split along non-vowels, with spaces removed. This seems to consistently slightly over-estimate the number of syllables, so we need to adjust it down (about .98 seems to do it):

y=r('[^aeiou](?i)+',i);y=len(y)-y.count(' ')-2;

202 -> 194: len(x)-2 rather than len(x[1:-1]). Removed unnecesary brackets. Made syllable regex case-insensitive

194 -> 188: The file was previously saved as dos rather than unix file format, causing wc -c to count newlines as two characters. Whoops.

188 -> 184: Get rid of those nasty x for x in ... if x!=...s by storing the intermediate result and subtracting x.count(...)

184 -> 171: Remove input/output, and convert to function

171 -> 167: Insert the len(x)-x.count(...)s into the formula

Tags:

Code Golf

Natural Language

Recent Posts