hoodwink.d enhanced
RSS
2.0
XHTML
1.0

RedHanded

Sparklines for Minimalists #

by why in inspect

If you’ve read Joe Gregorio’s Sparklines in data: URIs in Python, then you know that he’s presented us with an incredibly compelling use for data: URIs. Namely, this is not an external file: But we don’t have Python’s image library, so what are we to do?

What if I offered you a handful of code that could generate inline sparkgraphs without library code? Presenting Bumpspark.

 def bumpspark( results )
     white, red, grey = [0xFF,0xFF,0xFF], [0,0,0xFF], [0x99,0x99,0x99]
     ibmp = results.inject([]) do |ary, r|
         ary << [white]*15 << [white]*15
         ary.last[r/9,4] = [(r > 50 and red or grey)]*4
         ary
     end.transpose.map do |px|
         px.flatten!.pack("C#{px.length}x#{px.length%4}")
     end.join
     ["BM", ibmp.length + 54, 0, 0, 54, 40, 
       results.length * 2, 15, 1, 24, 0, 0, 0, 0, 0, 0].
       pack("A2ISSIIiiSSIIiiII") + ibmp
 end

It’s got one known bug: graphs built from 16-element arrays get staticky. Free inky duck drawing to first patcher. (Keep reading.)

Great coats! jzp has dropped a ping version in the comments! MenTaL has a bump one with compression! Peewee graphics libs within!

said on

Neat. I’ll have to play with this later if I get a chance.

Staticy images when you plot 16-element arrays is an interesting bug though.

Since I don’t have time to run through the code right now I’ll just pass along my gut feeling: I think you’re looking at byte alignment issues (perhaps due to rounding issues, with 16 being a likely boundary case). May also want to double-check the alignment requirements for rows imposed by the BMP format.

said on

Incidentally, the data: URIs don’t work in IE. So if you’re getting a broken image…

said on

Hrm, I had a few minutes so I tried it and I couldn’t reproduce the bug. Could you provide a sample results array for which it fails?

said on

The bitmap output size makes me cry. PNG ’s are so much smaller.

I thought about writing a PNG out-put-er – but I don’t want to deal with CRC ’s.

said on

You do rock though.

said on

Hmm, the resulting bitmaps are a bit in the large size, I suppose.

This should help a little bit (as well as make the code portable to non-little-endian architectures):


 def bumpspark( results )
     white, red, grey = 0, 16, 32
     ibmp = results.inject([]) do |ary, r|
         ary << [white]*15
         ary.last[r/9,4] = [(r > 50 and red or grey)]*4
         ary
     end.transpose.map do |px|
         px.pack("C#{px.length}x#{px.length&1}")
     end.join
     ["BM", ibmp.length + 66, 0, 0, 66, 40,
       results.length * 2, 15, 1, 4, 0, 0, 0, 0, 3, 0,
       0xFFFFFF, 0xFF0000, 0x999999 ].
       pack("A2Vv2V4v2V9") + ibmp
 end
said on

Incidentally, the blaahg preview eats my plus signs.

said on

Innocent bystander I am, clappings hands red.

said on

You could get around the IE problem by making it an HTML image and making the url point to a ruby script that returned a gif or png mime type and the actual byte content of the image.

said on

The following hack on the original encodes the URLs in base64, and creates compressed PNGs for extra space saving.

Like MenTaLguY, my plus signs seem to go away in the preview.


require 'base64'
require 'cgi'
require 'zlib'
def build_png_chunk(type,data)
    to_check = type + data
    return [data.length].pack("N") + to_check + [Zlib.crc32(to_check)].pack("N")
end
def build_png(image_rows)
    header = [137, 80, 78, 71, 13, 10, 26, 10].pack("C*")
    raw_data = image_rows.map { |row| [0] + row }.flatten.pack("C*")
    ihdr_data = [   image_rows.first.length,image_rows.length,
                    8,2,0,0,0].pack("NNCCCCC")
    ihdr = build_png_chunk("IHDR", ihdr_data)
  idat = build_png_chunk("IDAT", Zlib::Deflate.deflate(raw_data))
    iend = build_png_chunk("IEND", "")

    return header + ihdr + idat + iend
end
def bumpspark( results )
     white, red, grey = [0xFF,0xFF,0xFF], [0,0,0xFF], [0x99,0x99,0x99]
     rows = results.inject([]) do |ary, r|
       ary << [white]*15 << [white]*15
         ary.last[r/9,4] = [(r > 50 and red or grey)]*4
         ary
     end.transpose
     return build_png(rows)
end
said on

Hmmn. Seem to have forgotten to include the URL encoding bit in the above post. Not sure if the CGI.escape ing is really necessary. The URL should be quoted with CGI.escapeHTML anyway before getting put inside double quotes in an img tag.


def build_data_url(type,data)
    data = Base64.encode64(data).delete("\n")
    return "data:#{type};base64,#{CGI.escape(data)}" 
end
url = build_data_url("image/png",bumpspark(results))
said on

Carl, that’s wonderfully devious.

jzp, that’s excellent. I had forgotten I could have had zlib at my fingertips. While you’re at it, why not make an indexed PNG ?

said on

As a matter of interest, I tried a version that produced RLE4 -compressed bitmaps (the sparklines are well-suited to that, since they skip every other row and thus always let us combine every two pixels).

PNG consistently wins over that at half the size.

said on

SVG ?

said on

MenTaLguY is now the overlord for RedHanded comments. I think I’m going to add a second layer of comments (to the right hand of every comment) where MenTaLguY can annotate. I’m brutally serious.

Krikey, jzp! Send me your address. It’s true that you get a duck. (MenTaL, send me yours too. You get a clock.)

Next up: we gotta make a 1k graphics lib for Ruby out of this.

said on

Can we get the .pngs to have transparent backgrounds instead of white?

said on

Yes, a transparent background is quite easy; this is untested, but I believe you need only modify build_png thusly:


def build_png(image_rows)
    header = [137, 80, 78, 71, 13, 10, 26, 10].pack("C*")
    raw_data = image_rows.map { |row| [0] + row }.flatten.pack("C*")
    ihdr_data = [   image_rows.first.length,image_rows.length,
                    8,2,0,0,0].pack("NNCCCCC")
    ihdr = build_png_chunk("IHDR", ihdr_data)
  trns = build_png_chunk("tRNS", ([ 0xFF ]*6).pack("C6"))
  idat = build_png_chunk("IDAT", Zlib::Deflate.deflate(raw_data))
    iend = build_png_chunk("IEND", "")

    return header + ihdr + trns + idat + iend
end

The added tRNS chunk indicates that white ( 0xFFFF, 0xFFFF, 0xFFFF ) is to be interpreted as transparent.

said on

Concerning jzp’s pung stuff: The sparklines are coming out upside down, unless I do a transpose.reverse. The transparency fix looks right, but it’s not right. Tinker tinker.

said on

Re: transparency: it’s also possible to use RGBA pixel values instead of RGB and have per-pixel alpha. Although, I don’t know whether any browsers actually render these properly.

Re: upside down: Oops! I should have tested more…

Re: indexed PNG : Haven’t tried it yet. I’m hoping that the compression of RGB pixels does a fairly good job. However, I suppose 4-bit indexes could reduce the data nicely. Makes packing a bit more complicated though!

I did try implementing the Up() filter which does a line-by-line delta (good since there’s lots of vertical redundancy in these sparklines), but the compressed data actually got bigger in my little test case (!), so I abandoned it.

said on

Cant you do the same things with inline XBitmaps in HTML ?

said on

As a red Tufte disciple, I find myself prostate to this sparkline development.

But, “honey, where’s my super test/unit suit?”

said on

Requesting the continuous plot in Joe Gregorio’s updates.

said on

This is good stuff. I think it would be great for adding barcodes to pages (though most readers emit light, so won’t read off the screen). But reading code off here in the narrow column for comments is tricky for me, even when I make the font too small for comfort. _Why, please could you copy the comments to your Bumpspark page, for easier access? Thank you.

said on

hgs: The latest working ping spark is up on the Bumpspark page now. I haven’t been able to get MenTaL’s compressed bump to work yet. It’s close, but just not ready yet.

said on

Thank you, that’s much easier to see. I may have something to contribute later…

said on

Mine (the one I posted at least) isn’t really compressed; it just creates a 4bpp indexed BMP rather than a 24bpp truecolor one.

Out of curiousity, what problems are you having with with that version? I didn’t test it exhaustively, but I thought I checked all the corner cases.

said on

Whoa, never mind. I see what you mean.

Argh… I think I uploaded the wrong version. And I’d just deleted my scratch files since I thought I’d preserved the final version for posterity here….

said on

I love the pingspark! So much information in so few bytes.

red should be [0xFF,0,0] not [0,0,0xFF] in pingspark though.

said on

Yeah, stride issues. I remember now. It’s the x#{px.length&1} bit.

I don’t remember what the correct solution is. The alignment rules were a bit non-obvious.

said on

If I’m correct Base64 encoded data does not need to be cgi escaped for data urls. It’s at least working well for me without the extra escaping.

Base 64 is just: 26 lower case letters 26 upper case letters 10 digets 1 ”+” 1 ”/”

64 characters.

said on

Ah, I’m stumped. Even simple byte-alignment doesn’t work. Check this (using a 4bpp bumpspark with configurable byte alignment for rows): width/alignment matrices

said on

Sparklines in use.

http://www.braino.org/blog/images/sparklines.gif

said on

Oh. Duh. Look at the 4-byte alignment column.

I suspect that might start working if an extra byte of padding were added to the end of the odd-byte-width ones.

Let me try that…

said on

(I have a feeling I never really had it quite working before, I was just “lucky” in picking my test cases)

said on

Got it!


 def bumpspark( results )
     white, red, grey = 0, 16, 32
     padding = 3 - ( results.length - 1 ) % 4
     ibmp = results.inject([]) do |ary, r|
         ary << [white]*15
         ary.last[r/9,4] = [(r > 50 and red or grey)]*4
         ary
     end.transpose.map do |px|
         px.pack("C#{px.length}x#{padding}")
     end.join
     ["BM", ibmp.length + 66, 0, 0, 66, 40,
       results.length * 2, 15, 1, 4, 0, 0, 0, 0, 3, 0,
       0xFFFFFF, 0xFF0000, 0x999999 ].
       pack("A2Vv2V4v2V9") + ibmp
 end
said on

Regarding data not workin in IE: I think you can instead use a javascript:’data, it\’s going here’ style URLs to work around that. For images and so you might also have to overwrite the content-type which is possible when using instead of .

said on
Not bumpspark but nice too:

def fract( iter ) # use a small value like 15 as iter and wait a few seconds
    ycoord = (0..127).map{|i|(i/63.0)}
    xcoord = (0..255).map{|i|(i/63.5)-2.6}
    half = ycoord.map do |y|
      xcoord.map do |x|
        zr = zi = 0.0
        cr,ci = x,y
        i = 0
        while i < iter and (zr*zr + zi*zi)<8
          zr,zi = zr*zr-zi*zi+cr,2*zr*zi+ci 
          i+=1
        end
        if i == iter
          [0,0,0]
        else
          va = (zi.abs*127).to_i^(vb=zr.abs*127).to_i
          [[0,va,vb],[va,vb,0],[vb,0,va],[0,vb,va]][i&3]
        end
      end
    end
    build_png(half.reverse+half)
end
said on

alas, this inline data stuff makes Privoxy (my filtering proxy) die, wit a quickness.

said on

anyone else not able to see the data uri on this page in Firefox’s view source? On mine it just appears as white and I have to highlight it to see it. Seems to only be limited to the image on this page though. (filed bug 293875)

said on

For those looking for a less clever but more featureful RMagick implmentation, try my Sparkline Library for Ruby

And, it comes with a helper so you can use it with Rails in a cross-browser-friendly way!

said on

TiddlyWiki can build sparklines on the client side. Some helpers for doing the same thing in Rails would probably go a long way. Saves a lot of bandwidth, too, when you only have to pump the numbers down the pipe. :-)

said on

jjkllkjklljkljlkjlkj

said on

Does anybody know why some combinations don’t work? For example, this sequence of numbers gives me a blank image:

[15, 13, 61, 35, 79, 9, 59, 79, 73, 50, 82, 88, 14, 80, 81, 41, 43, 33, 3, 97, 75, 4, 42, 77, 46, 1]

Comments are closed for this entry.