Hibari News: Presentations in Chicago and Tokyo

As mentioned in an earlier blog posting, I was in Chicago on August 25th to talk to the Chicago Erlang User Group about the Hibari key value store.  (Not the manga character or the announced-a-few-weeks-ago Twitter application by the same name.)

The slides for my Chicago presentation were in two pieces.  The first set is available via SlideShare.  The second are available from my home Web server.

  1. First half: Hibari — Key-Value store for big data.  Note that these slides are actually newer than the ones that I showed in Chicago and include material that wasn’t available to me in Chicago.
  2. Second half: Hibari from a Developer’s Point of View.

Carl Karsten made a great recording of two hours (!) of the meeting.  You can find the streaming video by following this link to blip.tv.

Also in the same week, Gemini Mobile Technologies was busy with presentations in forums in Tokyo.  The “Key-Value store for big data” link above was one.  The other was Software, WebMail, BigData; Powered by Erlang & Functional Programming.

Finally, the CEUG video recording was mentioned quite prominently over at HighScalability: see Hot Scalability Links For Sep 3, 2010. Very spiffy.

    Posted in Erlang | Leave a comment

    2010 ACM Erlang Workshop approaching quickly

    September 30th is quickly approaching.  The 2010 ACM Erlang Workshop will be held on 9/30 in Baltimore, Maryland, not too far from Washington, DC.  There are plenty of seats still available, so make your registration and travel plans now.  :-)

    The URL http://www.erlang.org/workshop/2010/ has the full program & schedule. We’re happy to have David Smith from Basho Technologies as our invited speaker. The title of his talk is “Concurrent Life, the Universe, and Everything Rebar”

    We will also be adding 5-minute lightning talks at the end of the day, as many as we can fit in before they kick us out of the room at 6pm.

    The papers this year are excellent. Several of them are going to spark a lot of discussion & debate: at lunch, at session breaks, and over dinner and beer/whiskey/beverage-of-your-choice afterward.

    Also, I suggest that you look at the CUFP program (Commercial Users of Functional Programming) for lots of interesting tutorials and presentations.

    I look forward to seeing you in Baltimore!

    -Scott

    Workshop Chair, 2010 ACM Erlang Workshop

    Posted in Erlang | Tagged , | Leave a comment

    Hibari Presentation to Chicago Erlang User Group: Wednesday 25 August, 2010

    I’ll be giving a presentation about Hibari to the Chicago Erlang User Group (and hopefully some ChicagoDB folks) on Wednesday, 25 August, 2010 at 6pm.  Orbitz will be hosting the event, please see the CEUG event at MeetUp for more time and location details.  It looks like I’ll be arriving in O’Hare shortly after noon on Wednesday and will be returning Thursday morning.

    Here’s a copy of the abstract that I submitted for the talk:

    Hibari is a distributed, fault tolerant, highly available key-value store written in Erlang. Data replicas are managed by chain replication rather than the more common quorum voting technique used by Riak, Cassandra, and others. Hibari supports micro-transactions (atomic updates to multiple keys in limited circumstances) and cluster resizing (both growing and shrinking the cluster). Hibari clients can be written in Erlang as well as C, C++, Java, Python via the UBF protocol and soon in other languages via Protocol Buffers and Thrift.

    Hibari was released to the open source community under an Apache License v.2.0 and is available at http://hibari.sourceforge.net/. Hibari’s largest commercial deployment is a Webmail system in Asia that is managing approximately one billion email messages.

    My talk in Chicago will be the Hibari’s first public presentation. I’ll talk about its major features, how chain replication works, and how Hibari’s sponsor (Gemini Mobile Technologies) will manage code patches from the open source world. I’ll leave the rest to audience questions, with just-in-case material for demo’ing things like cluster expansion/contraction and developing a small client application that uses micro-transactions.

    Many thanks to Martin Logan, Chris Duesing, Eric Merritt, and Jordan Wilberding for the invitation and for all of the cat herding heroics that an active user group needs.

    Posted in Erlang | Tagged , | 1 Comment

    Hibari Database Released Under Apache Public License v2

    It was difficult to leave Gemini, a decision that I’d made back in May. I didn’t know if this day would arrive: would Gemini proceed with its plans to release the core of Hibari publicly? Once I was on the outside, I couldn’t see in again. Would there be another delay? Perhaps a change of business strategy?

    It’s now official: Gemini Mobile Technologies has release the Hibari key-value distributed database into the wild.  It is released under the Apache Public License version 2.

    The open source version is available at: http://sourceforge.net/projects/hibari/

    In total, I’m glad that Gemini has stuck to the (revised) schedule and pushed this code out the front door.  Like almost all code, it has some warts and things that need more spit & polish.  But there’s a tremendous amount of good stuff in there also.  At least report (about a month ago), Hibari was storing over 800 million email messages in a custom Webmail system (also developed by Gemini … see this presentation for some details.)

    I believe that Hibari is a good addition to the non-relational, non-traditional, distributed database ecosystem that has been evolving in the last ten or so years. The next ten years are going to be even more fun.  Welcome, Hibari!

    -Scott

    P.S. As far as I know, the build & packaging probably only works cleanly on Linux boxes (Fedora and CentOS specifically but hopefully most other distros). The packaging that I’d done (before leaving Gemini) for MacOS and FreeBSD doesn’t appear to be included in this initial release.

    For MacOS specifically, you’ll need to rename the “bom” script to “bom.sh” before executing it. Also, the “libtoolize” tool has been renamed “glibtoolize”. I created a simple 1-line Bourne/BASH shell script called “libtoolize” that simply uses ” exec glibtoolize $*” to execute the local version. And there’s a problem with compiling “start-stop-daemon.c”.  I expect that these OS/platform dependencies will be smoothed out over the next few weeks.

    Posted in Erlang | Tagged , | Leave a comment

    Final MSPIFF 2010: Monday May 3: USA, Ethiopia, Finland

    Well, it’s been a while since the MSPIFF “Best of the Fest” series finished.  I did manage to see three films that I’d wanted to catch but couldn’t, due to work conflicts or sleep or whathaveyou.  All three were on Monday, May 3rd:

    • No More Smoke Signals, USA.  This is a Swiss film about the Pine Ridge Indian Reservation in southwest South Dakota.  The central character is a radio station, KILI, 90.1MHz FM.  This radio station is the prairie home companion (lack of capital letters are intentional) and part cultural glue for the Oglala Lakota nation in Pine Ridge and elsewhere in South Dakota.  It’s run on a shoestring budget, including an eagle feather substitute for a more expensive lightning arrestor system.  The feather didn’t work, unfortunately: the station was blown off the air by a strike in 2006 and caused enough damage that it took over a year to raise the funds to fix the damage.  We see how the station knits the Lakota nation together as poverty and (the filmmaker suggests) the US government’s persecution of the American Indian Movement threaten to undo it.  Recommended.  May show again in the Twin Cities later in the summer? (MSPIFF link, includes link to trailer)
    • The Athlete, Ethiopia.  A based-on-the-true-story story about Abebe Bikila, the barefooted marathon runner who surprised the Olympic world in 1960 with his gold medal finish in Rome.  (MSPIFF link)  The start-from-the-middle beginning was a bit confusing, but things fell together quickly.  With so much resentment in Ethiopia toward Italy’s invasion in World War II, the entire nation was overjoyed by Bikila’s win in Rome.  “It took 500,000 Italian soldiers to conquer Ethiopia.  It took only one Ethiopian soldier to conquer Italy,” the saying goes now.  Bikila wins gold in 1964 and even more fame, especially since he’d had an appendectomy six weeks before The Games in Tokyo.  But injuries prevent him from finishing races in Mexico City and other high profile races … and those failures torment him.  A car accident leaves him unable to walk, much less win.  But Bikila is one amazingly tough, competitive son-of-a-gun.  Over a third of the crowd in the theater were east African, and their cheers were as infectious as the movie itself.  The director’s very occasional use of news & film clips from the 60′s and 70′s of Bikila himself were, I think, very effective.  Recommended.  There’s talk that it may get a distribution deal in the US for later this year?
    • Tears of April, Finland.  (MSPIFF link) The civil war in Finland at the end of World War I was quite a whopper.  The nationalist Whites defeated the Russian-supported Reds, who were then hunted and killed in revenge.  One principled lieutenant refuses orders to shoot “fleeing” prisoners and instead brings the lone survivor to a judge for trial.  He’s a noted author and poet, an intellectual man who’ll give her a fair trial.  But war has sunk its claws deep into Finland, and the judge and his staff are not immune.  Not a happy ending, but then again, how many civil wars do?  The Cekov/study of the asylum/One Flew Over the Cookoo’s Nest themed bits were too thick, but they’re tolerable.  Recommended.

    So, that’s it.  I may follow with a follow-up to this follow-up, since the festival’s audience poll numbers were printed and distributed during the Best of Fest period.  When/if I do that, it’ll be another 11 months before the next Minneapolis/St. Paul International Film Festival.  {sniffle}  I had a blast and am most definitely looking forward to next year.

    Posted in Film | Tagged , | Leave a comment

    Hibari update: still writing documentation

    I don’t have any big news yet to announce regarding Hibari (Gemini link).  We’re still waiting on some negotiations to finish before making a big public announcement.  That announcement is planned for mid-May.  That’s still the goal, though it may slip to a bit later in the month in the worst case.

    Meanwhile, I’ve been writing a lot of documentation these past three weeks.  Between the “Sysadmin Guide” and the “Developer’s Guide”, I’ve got almost 130 pages of A4-formatted PDF files.  I still haven’t figured out how to do the typesetting of client API parts of the Dev Guide.  We’re using AsciiDoc to create PDF and HTML output, but AsciiDoc’s tool set doesn’t map well onto my problem.  {sigh}

    Posted in Erlang | Tagged , | 3 Comments

    MSPIFF: Thursday 4/29: Italy, USA

    Thursday was Day 14 of the Minneapolis/St. Paul International Film Festival, the last full day with 5 screens running.  It doesn’t seem like things have been running for two weeks now.  Louise was able to escape from grading long enough to dash to Punch for some tasty, tasty pizza and the first of these two films:

    • I am Love, Italy.  This film had got a 4-of-4 review from the Star Tribune, if I’m not mistaken.  Or perhaps word-of-mouth had spread from its early 4/25 showing?  Tickets were sold out.  Not even the power of my Gold Pass was entirely sufficient to get us two tickets: intervention by Fest co-organizer Ryan Oesterich, who happened to be standing next to the ticket counter when Louise and I were there, got us the second ticket.  Had we known that the film would start over half an hour late, we could have relaxed while eating pizza and beer over at Punch.  Oh well.  At least the Gold Pass superpower sprung us to the head of the line when seating finally did start.  Was the film (MSPIFF link) good?  Yup.  Was it worth 4 stars?  Both of use thought 3.5 was more fitting for the acting and writing.  If the Strib critic was swayed by the cinematography, I could believe that.  Recommended.  Not showing again.
    • TiMER, USA.  This is no-phaser-battles, no rogue-computers, no blue-skinned or green-blooded aliens science fiction movie.  What if science could provide you with a timer that would ring the instant you first made eye contact with your one true love in all the world?  What if you got your TiMER at age 14, and it told you that you’d meet Mr. Right at age 42?  What if your TiMER was blank, meaning that Mr. Right hadn’t purchased a TiMER yet?  How can you live without a guarantee of love?  I liked it enough to rate it Highly Recommended.  Not showing again, though hopefully this one will break through to wider distribution nationwide.

    It looks like I won’t be going to any films on Friday, the last day of the fest.  Looking at the “Best of Fest” schedule for May 1-6, I think I’ll be returning for a few more films.  NOTE: “The Girl” from Sweden won the viewer’s balloting for best film at MSPIFF this year.  It’s showing on Saturday May/1 at 9:30pm.

    Posted in Film | Tagged , | Leave a comment

    MSPIFF: Wednesday 4/28: Austria, USA

    Wednesday was Day 13 of the Minneapolis/St. Paul International Film Festival. It was a day of contrasts, the concrete and the surreal.

    • Pianomania, Austria.  This documentary follows a year in the professional life of Stefan Knüpfer, a piano technician and tuner who works in concert halls and venues in Austria.  (MSPIFF link) His pursuit of the ideal piano sound is truly geeky, in large part because “ideal” depends on the pianist, the music, the piano (we learn quite a bit about Steinway serial numbers #109 and #245), the piano’s many parts, and the concert hall.  Near the end, we get to see the music duo Igudesman & Joo (of “Rachmaninoff Had Big Hands” fame) work with Knüpfer on some new jokes.  Recommended.  Not showing again.
    • Presque Isle, USA.  Weird, dreamy, surreal, dreamy, what-the-****, arty.  And a bit cool at the same time.  Skip.  Not showing again.
    Posted in Film | Tagged , | Leave a comment

    MSPIFF: Tuesday 4/27: Canada, USA, and Best of Fest Schedule

    Tuesday was Day 12 of the Minneapolis/St. Paul International Film Festival. Mom left for home today, so I didn’t have a movie buddie today.

    • Father and Guns, Canada.  I was expecting a comedy, and I got one.  “De père en flic” (MSPIFF link) is the highest-grossing French Canadian film.  What I didn’t expect was the acting.  There are some serious, emotional moments throughout the film, and the film had my attention and everyone else’s in the theater.  No giggles, no snide “yeah, right” comments.  And the funny bits were pretty damn funny. Recommended. Showing again on 4/29.
    • Dear Lemon Lima,, USA.  Coming of age genre, check.  Struggles with infatuation and heartbreak, check.  Outsiders and geeks, check.  Misfits hanging out in detention after school (Breakfast Club) the weightroom during gym class (MSPIFF link), check.  “Napoleon Dynamite” meets “Juno“?  Well, without the pregnancy, maybe.  It was sweet, definitely quirky, and safe/no-bad-words-or-images for young families, if that’s a worry.  Recommended.  Not showing again.

    The “Best of Fest” schedule is now available for Saturday – Thursday, May 1-6.  I hope to catch “La Mission” and “The Athlete” and “The Boneman” and “North” during that week, oh oh, and “No More Smoke Signals”.  If you’re a documentary nut, these three are showing again during the Best and they’re all worth seeing (see my previous blob postings): “The Unreturned”, “Ghosts of the 7th Cavalry”, and “Pink Taxi”.

    Posted in Film | Tagged , | Leave a comment

    MSPIFF: Monday 4/26: Canada, Italy

    Monday was Day 11 of the Minneapolis/St. Paul International Film Festival. Mom stayed with us for a few days, to catch up on the worst of jet lag before driving home on Tuesday.  Movies are a great cure for jet lag….

    • Genius Within: The Inner Life of Glenn Gould, Canada.  I’d known a bit about Glenn Gould, but this bio-pic (MSPIFF link) helped explain why he was such a sensation in his day.  His move away from live performances to the recording studio was amazingly prescient: he predicted over 30 years ago that people would be recording and mixing audio at home.  He thought it would be using equipment kits shipped to homes across North America.  Instead, it came in the form of personal computers.  Highly recommended.  Not showing again.
    • The Sicilian Girl, Italy.  Based on the true story of Rita Atria, a 17-year old young woman who has kept diaries for over five years of the Sicilian Mafia’s activities in her village.  She turns the diaries over to the state’s chief prosecutor to get revenge for the murder of her father and her brother.  In the end, she’ll settle for justice.  This story does not have a happy ending, but it demonstrates the Mafia’s corruption of Sicily in many ways that Franco used to keep Spain under his control.  (See “Forever Waiting” in an earlier blog posting.)  Recommended.  Not showing again.
    Posted in Film | Tagged , | Leave a comment