RSS clients, please, stop pinging every 60 seconds

Hi, everyone. I know that y’all is simply craving to hear everything that I have to say, as quickly as I say it.  That’s why you use an RSS client, to keep track of my blog.  You keep tabs on me, because I’m worth keeping tabs on.  I’m so worth it that your RSS client is printing my blog’s RSS feed every 60 seconds.

I might say something. At any time.

From this day forward, I’ll separate my blog postings by several hours, if not days or months.  You won’t miss anything, really, if you polled less frequently.  Besides, you’re probably a Twitter user.  Follow me on Twitter, @slfritchie, to get your fix.

My firewall might say something.  To you, 50.16.238.123.  At any time.

Except that you’re likely running on an EC2 instance.  Fickle, mercurial.  Poised to multiply a thousandfold like tribbles, overruning my modest ISP service, flooding its virtual storage bins with cute replicas of their fuzzy terror.

The firewall “man” page says nothing about filtering TDP, the Tribble DoS Protocol.  Drat.  I need an upgrade.  At an unknown time.

Posted in None-of-the-above | Tagged | 3 Comments

DTrace, FreeBSD 9.0, and Erlang

While working on the DTrace probes for the Erlang R15 release (which I’ve blogged about earlier this month), I discovered that the build recipe for FreeBSD 9.0 was broken.  Oh no, my favorite OS doesn’t work, sound the alarm!

FreeBSD has supported kernel-space DTrace probes for a while.  However, support for user-space probes, USDT, has not arrived until FreeBSD 9.0RC1.  “RC1” means “Release Candidate #1”.  As of this writing, RC2 is the newest release.  The official release of FreeBSD 9.0 (to be called FreeBSD 9.0-RELEASE) will not be ready for at least another month or perhaps longer; see the 9.0 wiki release schedule  for more details. Continue reading

Posted in Erlang | Tagged , , | Comments Off on DTrace, FreeBSD 9.0, and Erlang

SystemTap and Erlang: a tutorial

As mentioned in my previous posting, DTrace and Erlang: a status report, it’s also possible to use Linux’s SystemTap to watch the inner workings of the Erlang virtual machine.  This is possible via a DTrace compatibility layer built in to SystemTap.

In theory, any Linux system that supports user-space SystemTap should work just fine. Continue reading

Posted in Erlang | Tagged , , | 2 Comments

DTrace and Erlang: a status report

I thought it’d be a good idea to write a status report about my work with Erlang, DTrace, and some SystemTap.

Quick, I Want an Overview and Some Slides….

I gave a presentation at the Erlang User Conference  in Stockholm a couple weeks ago.  The topic was about adding DTrace probes to the Erlang virtual machine.  The presentation slides (and soon video, I hope) are available online at the Erlang Factory website.  Click on the easel icon to fetch a PDF copy of the slides.  When the video is available, it will probably appear as an embedded video player/applet/thingie. Continue reading

Posted in Erlang | 4 Comments

Been a while since I’ve written

Life has been busy.  Very good, but also busy.

Professionally, things at Basho are very good.  It’s a great bunch of smart, ego-firmly-in-check folks.  Basho sent me to the Erlang User Conference  in Stockholm a couple weeks ago to present my work on adding DTrace probes to the Erlang virtual machine.  Flying from Stockholm to San Francisco was a very long Sunday, but I managed to survive.  The meetings in San Francisco with my fellow Bashonians were great, as was a lunch with the old crowd at Gemini Mobile in Foster City.

I’m fighting a bad head cold at the moment.  That has made work on DTrace & Erlang stuff difficult this week.  (More on DTrace in a later posting.)

We’re quite looking forward to the Thanksgiving break next week, then travel to Salt Lake City the following weekend to visit our new niece Autumn as well as budding musician Thai and proud parents Victor and Carla.  And perhaps someday we’ll make plans for Christmas….

Posted in None-of-the-above | Comments Off on Been a while since I’ve written

Blog Roused from Slumber: the MSPIFF 2011

Yep, I haven’t written for a while.  One excuse is that I’ve started using Twitter.  After being a Twitter skeptic for quite a while, but I guess I’ve decided that Twitter’s 140 character limit is indeed enough for some things.  For those three loyal readers of this blog, here is the link to my Twitter messages of near-infinite wisdom.

Meanwhile, it’s about time for the 2011 Minneapolis/St. Paul International Film Festival to start.  I bought my gold pass today and had a chance to visit with a couple of the organizers for a couple of minutes.  Almost all films have been chosen and scheduled, but Web site is still undergoing a lot of development. One missing features is easy browsing films by date & time.

The printed program will likely be done in a week or so.

I haven’t looked at the whole program yet, but (strangely enough) the title The Lutefisk Wars (IMDB) managed to jump out at me….

Posted in Film | Tagged | Comments Off on Blog Roused from Slumber: the MSPIFF 2011

Hibari mentioned in InfoWorld article

This article on Oct 25th mentions Hibari in the “Erlang” section. As a Basho guy nowadays, I’m very slightly sad that the author didn’t also mention Riak, but as a Hibari fan, I’m quite happy.

Seven Programming Languages on the Rise: From Ruby to Erlang, once niche programming language are gaining converts in today’s enterprise. http://infoworld.com/d/developer-world/7-programming-languages-the-rise-620

Posted in Erlang | Tagged , | 2 Comments

Setting Up Hibari from Scratch in a CentOS 5 Virtual Machine

The following is based on a message that I’d sent a few weeks ago to the hibari-questions-en mailing list over at SourceForge. Since then, I’ve streamlined things slightly and added instructions on how to set up a CentOS 5 (inside a VMware virtual machine, though using “real” hardware is certainly OK also) to act as the host for a Hibari server node. Continue reading

Posted in Erlang | Tagged , | Comments Off on Setting Up Hibari from Scratch in a CentOS 5 Virtual Machine

basho_bench for Hibari

Howdy, all. I’ve written a first draft of a basho_bench driver for Hibari. Instructions on how to use it follow.

NOTE: You’ll need Git and Mercurial in order to check out the basho_bench source.

You’ll also need the (very cool) statistics package R installed in order to create basho_bench’s graphs. For CentOS 5 users, I don’t have any good advice, sorry. For Mac OS, once I’d installed MacPorts, I think it was just a matter of “port install R”.

Now make a copy of the example config file:

cp examples/hibari.config ./my-hibari.config

… and edit it to reflect your local environment. Sorry, I don’t have any extensive documentation on it right now. There is basho_bench- specific documentation in the “docs” subdirectory. All of the Hibari- specific stuff is (hopefully clear?) in the comments of the example config file.

The ‘code_paths’ section of the config will require pointers to a bunch of code directories in the Hibari source. *ALSO*, there is one patch that you need to install in the …/hibari/src/erl-apps/gmt_util__HEAD directory. Download the patch file and put it in /tmp. The URL is: http://www.snookles.com/scott/hibari/gmt_config.erl.patch

Then apply it and rebuild using:

cd /your/path/to/hibari/src/erl-apps/gmt-util__HEAD
patch -p1 < /tmp/gmt_config.erl.patch
rm .bom*
cd ../../../
make

OK, now back to editing “./my-hibari.config”. Hopefully the comments are clear enough for you to figure out what to do. As a first experiment, I suggest that you ignore the “Erlang native client” and instead use the EBF client. Most of the entries in examples/hibari.config are good enough.

I suggest changing ‘duration’ to 1 minute and ‘concurrent’ to something small’ish like 10 (until you’re familiar with the tool). Change the code_paths to reflect where you’ve got the Hibari source. Then change the ‘hibari_servers’ list to point to your server(s). For a single node system, make only one entry in this list. :-)

Now, time to run the tool.

./basho_bench ./my-hibari.config

The output data will go into a local subdirectory called “tests”. When the run is finished, you’ll see something like:

=INFO REPORT==== 16-Sep-2010::01:21:31 ===
application: basho_bench
exited: stopped
type: permanent
Test completed after 1 mins.

The command “make results” will create a graph. The graph is a PNG file that is located in the same directory as the test results. Note that there is a symbolic link in the “tests” subdirectory that points to the most recent test run results. For example, “tests/20100916_012028”. That file can be viewed by most bitmap graphic image viewing utilities; for MacOS, I usually use:

open ./tests/current/summary.png

I’ll include a sample config and graph. The config is for a 10 minute run with 100 load generator threads. This isn’t a very good test environment: the server is a small RAM, single disk machine. The basho_bench machine is a MacBook Pro that uses a WiFi network to talk to the server. This is not the kind of environment that you want to try to optimize, so don’t bother trying. This is just meant to be a quickie example, to be emailed before I fall asleep.

Happy hacking!

-Scott

(Click on the image below to see a full-size, hopefully-undistorted image).

Posted in Erlang | Tagged , | Comments Off on basho_bench for Hibari

Hibari News: Presentations in Chicago and Tokyo

As mentioned in an earlier blog posting, I was in Chicago on August 25th to talk to the Chicago Erlang User Group about the Hibari key value store.  (Not the manga character or the announced-a-few-weeks-ago Twitter application by the same name.)

The slides for my Chicago presentation were in two pieces.  The first set is available via SlideShare.  The second are available from my home Web server.

  1. First half: Hibari — Key-Value store for big data.  Note that these slides are actually newer than the ones that I showed in Chicago and include material that wasn’t available to me in Chicago.
  2. Second half: Hibari from a Developer’s Point of View.

Carl Karsten made a great recording of two hours (!) of the meeting.  You can find the streaming video by following this link to blip.tv.

Also in the same week, Gemini Mobile Technologies was busy with presentations in forums in Tokyo.  The “Key-Value store for big data” link above was one.  The other was Software, WebMail, BigData; Powered by Erlang & Functional Programming.

Finally, the CEUG video recording was mentioned quite prominently over at HighScalability: see Hot Scalability Links For Sep 3, 2010. Very spiffy.

    Posted in Erlang | Comments Off on Hibari News: Presentations in Chicago and Tokyo