<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<div class="moz-cite-prefix">For now, I am using the time honored
hack of a sleep(). I'm not sure if the runInBackground() is
necessary, but since the sleep()s make this take substantially
longer, that seemed like a reasonable thing to try.</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Any thoughts on a better answer?<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix"><font face="monospace">diff --git
a/app/Console/Commands/Irrdb/UpdateAsnDb.php
b/app/Console/Commands/Irrdb/UpdateAsnDb.php<br>
index 5332a0ec4..a91e76ead 100644<br>
--- a/app/Console/Commands/Irrdb/UpdateAsnDb.php<br>
+++ b/app/Console/Commands/Irrdb/UpdateAsnDb.php<br>
@@ -67,7 +67,13 @@ class UpdateAsnDb extends UpdateDb<br>
<br>
$customers = $this->resolveCustomers();<br>
<br>
+ $first = true;<br>
foreach( $customers as $c ) {<br>
+ if ($first) {<br>
+ $first = false;<br>
+ } else {<br>
+ sleep(1);<br>
+ }<br>
$task = new UpdateAsnDbTask( $c );<br>
$this->printResults( $c, $task->update(),
'asn' );<br>
}<br>
diff --git a/app/Console/Commands/Irrdb/UpdatePrefixDb.php
b/app/Console/Commands/Irrdb/UpdatePrefixDb.php<br>
index d75a892c4..dc5fbd847 100644<br>
--- a/app/Console/Commands/Irrdb/UpdatePrefixDb.php<br>
+++ b/app/Console/Commands/Irrdb/UpdatePrefixDb.php<br>
@@ -67,7 +67,13 @@ class UpdatePrefixDb extends UpdateDb<br>
<br>
$customers = $this->resolveCustomers();<br>
<br>
+ $first = true;<br>
foreach( $customers as $c ) {<br>
+ if ($first) {<br>
+ $first = false;<br>
+ } else {<br>
+ sleep(1);<br>
+ }<br>
$task = new UpdatePrefixDbTask( $c );<br>
$this->printResults( $c, $task->update(),
'prefix' );<br>
}<br>
diff --git a/app/Console/Kernel.php b/app/Console/Kernel.php<br>
index f00333073..b7ca9b90c 100644<br>
--- a/app/Console/Kernel.php<br>
+++ b/app/Console/Kernel.php<br>
@@ -57,9 +57,11 @@ class Kernel extends ConsoleKernel<br>
// IRRDB - <a class="moz-txt-link-freetext" href="https://docs.ixpmanager.org/features/irrdb/">https://docs.ixpmanager.org/features/irrdb/</a><br>
if( config( 'ixp.irrdb.bgpq3.path' ) &&
is_executable( config( 'ixp.irrdb.bgpq3.path' ) ) ) {<br>
$schedule->command( 'irrdb:update-prefix-db'
)->cron( '7 */6 * * *' )<br>
+ ->runInBackground()<br>
->skip( function() { return env(
'TASK_SCHEDULER_SKIP_IRRDB_UPDATE_PREFIX_DB', false ); } );<br>
<br>
$schedule->command( 'irrdb:update-asn-db'
)->cron( '37 */6 * * *' )<br>
+ ->runInBackground()<br>
->skip( function() { return env(
'TASK_SCHEDULER_SKIP_IRRDB_UPDATE_ASN_DB', false ); } );<br>
}<br>
<br>
</font><br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">On 2023-06-07 00:20, Richard Laager via
ixpmanager wrote:<br>
</div>
<blockquote type="cite"
cite="mid:c71f79ca-6796-bc1d-7aa9-619b8002e736@wiktel.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<p>I seem to be getting rate limited by RADB with update-asn-db /
update-prefix-db. It will get partway through and then die.
Doubly unfortunate is the fact that IXP Manager doesn't keep
track of where it was in the cache run, so the next time it
starts over again at the top. As a result, networks farther down
probably never get updated.</p>
<p>Are other people running into this?<br>
</p>
<p><br>
</p>
<p><a moz-do-not-send="true"
href="https://www.radb.net/support/informational/query.html"
class="moz-txt-link-freetext">https://www.radb.net/support/informational/query.html</a>
says:<br>
<br>
</p>
<blockquote type="cite">Please note: The RADb WHOIS service is
rate-limited. For large queries, we recommend using the RADb API<br>
<br>
Programmatic access is considered abuse of the service.</blockquote>
<br>
Isn't that exactly what IXP Manager / bgpq3 / bgpq4 are doing???
<p><br>
Later, it says:<br>
<br>
</p>
<blockquote type="cite"><span style="white-space: pre-wrap; display: block; width: 98vw;">If you plan on making a large number of queries please invoke a persistent TCP/IP session. This is done by telnetting directly to whois.radb.net and issuing the !! command. This will spare our server having to establish and teardown connections for every query.</span></blockquote>
<br>
bgpq4, at least, does that. But IXP Manager is still separately
invoking bgpq4 for each AS when running update-asn-db /
update-prefix-db.</blockquote>
<pre class="moz-signature" cols="72">--
Richard</pre>
</body>
</html>