Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I do a lot of very low level programming with awful performance-maintenance trade-offs. Here's a great trick for a "binary" JSON: remove all of the extra whitespace, normalize your numbers, and the LZ4 the resulting string.

UTF-8 is already a great wire format.

I've never found a "binary JSON" that's significantly better than this; I mean you can beat it, but you need awkward encodings (prefix indices & other weird shit). You end up burning nearly-byte for any particularly clever integer encoding.

Most data structures are just nested arrays of integers. If you need an integer keyed OBJECT you're SOL, but I just play fiddly games with astral plane UTF-8 characters. (Yeah yeah yeah ad hoc encodings are nasty news.)

If you've got a BUTT LOAD of data just fire up a compressing SQLite DB like a normal human.



If you're interested in performance, what about all the number conversion (to decimals, presumably) that is incurred with JSON?


If I'm interested in performance I'll build my data out of offset handles and lay everything down into a block and mmap() it around. That's parsing free, up to an htons() — but that's only a worst case scenario. Everything else is about not inventing something custom & being able to use easily vendored high-trust 3rd party tools. (In this case: a JSON library, LZ4, and/or SQLite.)




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: