Comments for Humans and Hippos in Harmony http://www.snookles.com/slf-blog Scott Lystig Fritchie's whimsy about computers, film, music, and whatever else his crazy neurons come up with that cannot fit into a Twitter tweet (@slfritchie). Thu, 19 Jan 2012 16:44:19 +0000 hourly 1 http://wordpress.org/?v=3.3.1 Comment on TCP incast: What is it? How can it affect Erlang applications? by slfritchie http://www.snookles.com/slf-blog/2012/01/05/tcp-incast-what-is-it/comment-page-1/#comment-7378 slfritchie Thu, 19 Jan 2012 16:44:19 +0000 http://www.snookles.com/slf-blog/?p=220#comment-7378 Mark, no, I haven't done anything with Linux 3.2. The machines most directly involved (and still involved) aren't "mine", they're customer boxes. Many thanks for the LWN link. I've been a long-time LWN subscriber, but I overlooked that article. -Scott Mark, no, I haven’t done anything with Linux 3.2. The machines most directly involved (and still involved) aren’t “mine”, they’re customer boxes.

Many thanks for the LWN link. I’ve been a long-time LWN subscriber, but I overlooked that article.

-Scott

]]> Comment on TCP incast: What is it? How can it affect Erlang applications? by Mark Wolfe http://www.snookles.com/slf-blog/2012/01/05/tcp-incast-what-is-it/comment-page-1/#comment-7358 Mark Wolfe Wed, 18 Jan 2012 21:15:25 +0000 http://www.snookles.com/slf-blog/?p=220#comment-7358 Great review of this issue, the background discussion really taught me a lot. I am interested to know whether you have tested the changes to TCP introduced in 3.2 of the linux kernel. I believe there were some changes to the logic of window size adjustment in cases where there was packet loss. There are some <a href="https://lwn.net/Articles/458610/" rel="nofollow">LPC: Making the net go faster</a> has a good explanation of the issue I am talking about. Thanks again for your post it taught me some great lessons. Great review of this issue, the background discussion really taught me a lot.

I am interested to know whether you have tested the changes to TCP introduced in 3.2 of the linux kernel. I believe there were some changes to the logic of window size adjustment in cases where there was packet loss.

There are some LPC: Making the net go faster has a good explanation of the issue I am talking about.

Thanks again for your post it taught me some great lessons.

]]> Comment on DTrace and Erlang: a status report by Some Guy http://www.snookles.com/slf-blog/2011/11/19/dtrace-and-erlang-a-status-report/comment-page-1/#comment-7350 Some Guy Wed, 18 Jan 2012 14:14:50 +0000 http://www.snookles.com/slf-blog/?p=159#comment-7350 This is a -major- addition. I was just wishing today that Erlang had a DTrace provider -- staring at truss output is a very poor substitute for what DTrace can do when given the right provider, but right now DTrace has little insight into Erlang's internals. So thanks for this work! This is a -major- addition.

I was just wishing today that Erlang had a DTrace provider — staring at truss output is a very poor substitute for what DTrace can do when given the right provider, but right now DTrace has little insight into Erlang’s internals.

So thanks for this work!

]]> Comment on TCP incast: What is it? How can it affect Erlang applications? by slfritchie http://www.snookles.com/slf-blog/2012/01/05/tcp-incast-what-is-it/comment-page-1/#comment-7092 slfritchie Sun, 08 Jan 2012 23:28:33 +0000 http://www.snookles.com/slf-blog/?p=220#comment-7092 Dan, we've compensated by changing a config knob within the Erlang virtual machine that creates something very similar to Gettys's buffer bloat. It doesn't help point-to-point latency between nodes A & B, but it can prevent premature descheduling of Erlang processes so that messaging between nodes A & C can proceed. Dan, we’ve compensated by changing a config knob within the Erlang virtual machine that creates something very similar to Gettys’s buffer bloat. It doesn’t help point-to-point latency between nodes A & B, but it can prevent premature descheduling of Erlang processes so that messaging between nodes A & C can proceed.

]]> Comment on TCP incast: What is it? How can it affect Erlang applications? by slfritchie http://www.snookles.com/slf-blog/2012/01/05/tcp-incast-what-is-it/comment-page-1/#comment-7091 slfritchie Sun, 08 Jan 2012 23:25:16 +0000 http://www.snookles.com/slf-blog/?p=220#comment-7091 Leebert, it's quite difficult to find commodity data centers that will offer that kind of data networking. My understanding is that if additional networking options are offered, bonding dual GigEs together is most common, then followed perhaps by 10GigE. Leebert, it’s quite difficult to find commodity data centers that will offer that kind of data networking. My understanding is that if additional networking options are offered, bonding dual GigEs together is most common, then followed perhaps by 10GigE.

]]> Comment on TCP incast: What is it? How can it affect Erlang applications? by Dan Creswell http://www.snookles.com/slf-blog/2012/01/05/tcp-incast-what-is-it/comment-page-1/#comment-7084 Dan Creswell Sun, 08 Jan 2012 16:30:11 +0000 http://www.snookles.com/slf-blog/?p=220#comment-7084 "But if your Ethernet switch has a lot of buffer space (like the HP or Juniper switches in the Tolly figure), you do not make it impossible to overrun your buffer space: large buffers only make buffer overruns less likely. Remember, these are fast machines in this cluster. And your switch may only have buffer space for a few dozen or a few hundred frames, depending on the frame size." As you're broadening your knowledge of TCP and network switches, you may also want to check out the phenomenon known as Buffer Bloat (the result of overly large buffers): http://gettys.wordpress.com/ “But if your Ethernet switch has a lot of buffer space (like the HP or Juniper switches in the Tolly figure), you do not make it impossible to overrun your buffer space: large buffers only make buffer overruns less likely. Remember, these are fast machines in this cluster. And your switch may only have buffer space for a few dozen or a few hundred frames, depending on the frame size.”

As you’re broadening your knowledge of TCP and network switches, you may also want to check out the phenomenon known as Buffer Bloat (the result of overly large buffers): http://gettys.wordpress.com/

]]> Comment on TCP incast: What is it? How can it affect Erlang applications? by leebert http://www.snookles.com/slf-blog/2012/01/05/tcp-incast-what-is-it/comment-page-1/#comment-7083 leebert Sun, 08 Jan 2012 15:43:16 +0000 http://www.snookles.com/slf-blog/?p=220#comment-7083 Oh, google for "flat network neighborhood" ... first result that jumped up & hit me: http://aggregate.org/FNN/ Oh, google for “flat network neighborhood” … first result that jumped up & hit me:
http://aggregate.org/FNN/

]]> Comment on TCP incast: What is it? How can it affect Erlang applications? by leebert http://www.snookles.com/slf-blog/2012/01/05/tcp-incast-what-is-it/comment-page-1/#comment-7082 leebert Sun, 08 Jan 2012 15:41:38 +0000 http://www.snookles.com/slf-blog/?p=220#comment-7082 There is an inexpensive hardware workaround to network saturation, although it can be a pain to set up. It used to be called a "local network topology" and has been used in supercomputer clusters - 4 NIC's per machine, with simple switches for neighborhood nodes, upstream from either the switch or 1st NIC (can't recall which). It's spaghetti but you won't saturate as soon. There is an inexpensive hardware workaround to network saturation, although it can be a pain to set up. It used to be called a “local network topology” and has been used in supercomputer clusters – 4 NIC’s per machine, with simple switches for neighborhood nodes, upstream from either the switch or 1st NIC (can’t recall which). It’s spaghetti but you won’t saturate as soon.

]]> Comment on Hibari mentioned in InfoWorld article by slfritchie http://www.snookles.com/slf-blog/2010/10/26/hibari-mentioned-in-infoworld-article/comment-page-1/#comment-7068 slfritchie Sun, 08 Jan 2012 01:39:59 +0000 http://www.snookles.com/slf-blog/?p=141#comment-7068 Don, it depends on the need for your app. For Gemini's Webmail app, we likely would not have been able to meet the development deadline if we had to manage the extra complexity of a non-strictly-consistent system. The customer's availability requirements were compatible with a strictly-consistent system, so ... Hibari was a good choice. -Scott Don, it depends on the need for your app. For Gemini’s Webmail app, we likely would not have been able to meet the development deadline if we had to manage the extra complexity of a non-strictly-consistent system. The customer’s availability requirements were compatible with a strictly-consistent system, so … Hibari was a good choice.

-Scott

]]> Comment on TCP incast: What is it? How can it affect Erlang applications? by slfritchie http://www.snookles.com/slf-blog/2012/01/05/tcp-incast-what-is-it/comment-page-1/#comment-7066 slfritchie Sun, 08 Jan 2012 01:23:52 +0000 http://www.snookles.com/slf-blog/?p=220#comment-7066 Anton, Riak attempts to fetch everything from all N replicas. -Scott Anton, Riak attempts to fetch everything from all N replicas.

-Scott

]]>