[ixpmanager] SFLOW Under Reporting?

[Ian Chilton]

Hi André,

Thank you for replying!

Comments below.

On 2023-06-22 14:32, André Grüneberg wrote:

> Are you by any chance using vlan translation or L2 sub-interfaces (with 
> non default VLAN IDs) on your Arista gear?

We have just started using L2 sub-interfaces in the last few months and 
it seems this has been a problem for longer for this.

We're aware that traffic on sub-interfaces won't be counted, but that 
only accounts for a fraction of the discrepancy we are seeing.

In addition, the member who was reporting inconsistency with their ports 
and their peers are not using sub-interfaces so that's not a factor 
there.

> In these cases the sFlow packets contain the VLAN ID on the wire and 
> will not be matched into the right buckets.
> We "enhanced" the sflow collector script with some hack to map the VLAN 
> ID. [I may go into details]

As I say, a different problem, but one that's on my list to fix so would 
be interested in what you did here.

Presumably it's just a case of extracting VLAN -> VLAN mappings of 
subinterfaces and substituting that in sflow data as it's processed?

> As far as I can see, you also offer private VLANs. These may also 
> account for some discrepancy between peering VLAN and overall traffic. 
> Unfortunately the private VLAN sFlow statistics are not (correctly) 
> exposed in P2P.

Again, not a factor with this member, but am interested if you've done 
any work to resolve?

> Besides those common issues, it also took us some effort to validate 
> that all peers are being counted "correctly".

Interested to hear more about how you did this? - and did you have to 
make any further changes to anything?

> We believe that our results (https://www.bcix.de/ixp/statistics/vlan) 
> are very close to reality.

Interesting! - so right now you're doing 507Gbps according to MRTG and 
showing 349G (v4) + 72G (v6) = 421G with sflow.

Are you using Arista too? - what sample rate?

I have just found another smoking gun - when running 
sflow-to-rrd-handler with debug mode, I see a lot of dropped/rejected 
flows. Some (most?) of these seem to be sub-interfaces, but it turns out 
that some MACs are not in the discovered macs table, so I need to 
investigate that further, but now we are using MAC ACLs, we'd probably 
be better switching to configured macs.

Thanks,

Ian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.inex.ie/pipermail/ixpmanager/attachments/20230622/b18eaa50/attachment.htm>