[ixpmanager] SFLOW Under Reporting?

[Ian Chilton]

Hi Andre,

Good call!

I think it needs to be in addition to fixing the script, but i've done 
this too!

With the periodic processing pushed to a thread, we continue to see what 
looks like correct values (roughly 2x what we saw before), just with a 
memory leak, which we have temporarily worked around by restarting the 
script every few hours.

The graph is also a LOT smoother than before - we just occasionally get 
a short period of spikes - if we're lucky, the buffer increase might 
solve that!

https://portal.lonap.net/statistics/vlan

Thanks,

Ian

On 2023-06-29 08:52, André Grüneberg wrote:

> Hi,
> 
> I didn't know where exactly to jump into the conversation. So I'd take 
> the tail.
> I eventually followed another lead. What happens while the 
> sflow-to-rrd-handler script is flushing data into RRDs?
> Well, it's not reading from the input pipe fed by sflowtool. Ok, there 
> should be a buffer on the pipe?!
> But eventually sflowtool will block processing, because it cannot feed 
> data into the output pipe.
> On the other end of sflowtool, it receives UDP datagrams. Isn't there a 
> buffer? And indeed, Linux is buffering the inbound flow of UDP.
> 
> So I looked at `netstat -uanp | grep 6343` ... second column (Recv-Q) 
> show the use of the buffer. From time to time (during flush), the 
> buffer is filled ... and maxes out at ~200k.
> Next I increased the buffer size to 25M using sysctl:
> net.core.rmem_max=26214400
> net.core.rmem_default=26214400
> 
> After a restart of the script, I can see that the receive queue fills 
> up to ~10 MiB.
> 
> Looking at the graphs I see an improvement 
> (https://www.bcix.de/ixp/statistics/vlan/1/ipv4/bits, ~9:30 today).
> 
> Maybe this helps others?!
> 
> André
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.inex.ie/pipermail/ixpmanager/attachments/20230704/77627642/attachment.htm>