<div dir="ltr"><div>Hi,</div><div><br></div><div>I didn't know where exactly to jump into the conversation. So I'd take the tail.</div><div>I eventually followed another lead. What happens while the sflow-to-rrd-handler script is flushing data into RRDs?</div><div>Well, it's not reading from the input pipe fed by sflowtool. Ok, there should be a buffer on the pipe?!</div><div>But eventually sflowtool will block processing, because it cannot feed data into the output pipe.<br></div><div>On the other end of sflowtool, it receives UDP datagrams. Isn't there a buffer? And indeed, Linux is buffering the inbound flow of UDP.</div><div><br></div><div>So I looked at `netstat -uanp | grep 6343` ... second column (Recv-Q) show the use of the buffer. From time to time (during flush), the buffer is filled ... and maxes out at ~200k.</div><div>Next I increased the buffer size to 25M using sysctl:</div><div>net.core.rmem_max=26214400<br>net.core.rmem_default=26214400</div><div><br></div><div>After a restart of the script, I can see that the receive queue fills up to ~10 MiB.</div><div><br></div><div>Looking at the graphs I see an improvement (<a href="https://www.bcix.de/ixp/statistics/vlan/1/ipv4/bits">https://www.bcix.de/ixp/statistics/vlan/1/ipv4/bits</a>, ~9:30 today).<br></div><div><br></div><div>Maybe this helps others?!</div><div><br></div><div>André<br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, 28 Jun 2023 at 12:34, Nick Hilliard (INEX) via ixpmanager <<a href="mailto:ixpmanager@inex.ie" target="_blank">ixpmanager@inex.ie</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div><span>Ian Chilton wrote on 28/06/2023 07:54:</span><br>
<blockquote type="cite">Anyone have
experience with threads in Perl and able to suggest why?
<br>
</blockquote>
<br>
shorter answer: <a href="https://twitter.com/nedbat/status/194452404794691584" target="_blank">https://twitter.com/nedbat/status/194452404794691584</a><br>
<br>
Probably what's happening here is that there is some thread magic going
on behind the scenes which is creating anonymous references to functions
/ variable. If this happens in perl, and the anonymous reference is
lost, then garbage collection won't free up these resources until the
script exits. Although not very sexy, it would probably be safer to
fork a subprocess to handle the flush, rather than using threads. Even
this approach needs to be handled carefully (i.e. close the sflowtool
pipe for the child process, ensure the child process has a well defined
exit point, and put in some magic to ensure that orderly shutdowns
work).<br>
<br>
Nick<br>
<br>
</div>
_______________________________________________<br>
INEX IXP Manager mailing list<br>
<a href="mailto:ixpmanager@inex.ie" target="_blank">ixpmanager@inex.ie</a><br>
Unsubscribe or change options here: <a href="https://www.inex.ie/mailman/listinfo/ixpmanager" rel="noreferrer" target="_blank">https://www.inex.ie/mailman/listinfo/ixpmanager</a><br>
</blockquote></div><br clear="all"><br><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr">André Grüneberg, Managing Director<br><a href="mailto:andre.grueneberg@bcix.de" target="_blank">andre.grueneberg@bcix.de</a></div><div dir="ltr">+49 30 2332195 42<br><p>BCIX Management GmbH<br>Albrechtstr. 110<br>12103 Berlin<br>Germany</p><p>Geschäftsführer/Managing Directors: Jens Lietzmann, André Grüneberg<br>Handelsregister: Amtsgericht Charlottenburg, HRB 143581 B</p><font size="1"><span style="font-family:Calibri,"sans-serif"" lang="EN-US"></span></font></div></div></div></div></div></div></div></div></div></div>