<div dir="ltr">Rowan,<div><br></div><div>Thank you so much. I had ran out of time, and that was just the last bump in the road I had hit.</div><div><br></div><div>I was able to spend more time on this, and reading your script. I was able to glean that I was not using "Debian/Wezzy" so it was not checking all of the depencies.</div>
<div><br></div><div>I have ran them manually and am getting much further. I think I am almost all of the way through. I will have to spend more time on this later this weekend.</div><div><br></div><div>Cheers,</div><div>
Brian </div></div><div class="gmail_extra"><br clear="all"><div><strong style="font-family:'Lucida Grande',Verdana,Arial,sans-serif;font-size:12px;line-height:16px;color:rgb(136,136,136)">Brian Thompson</strong><br>
<span style="color:rgb(0,102,51);font-family:'Lucida Grande',Verdana,Arial,sans-serif;font-size:12px;line-height:16px">Senior Infrastructure Engineer // Senior Second Guesser</span><br><font color="#666666" face="Lucida Grande, Verdana, Arial, sans-serif"><span style="font-size:12px;line-height:15px"><br>
</span></font><span style="color:rgb(102,102,102);font-family:'Lucida Grande',Verdana,Arial,sans-serif;font-size:12px;line-height:16px">Direct: </span><a value="+15039436742" style="font-family:'Lucida Grande',Verdana,Arial,sans-serif;font-size:12px;line-height:16px;color:rgb(17,85,204)">503.943.6779</a><br>
<span style="color:rgb(102,102,102);font-family:'Lucida Grande',Verdana,Arial,sans-serif;font-size:12px;line-height:16px">Mobile: </span><a value="+15033196955" style="font-family:'Lucida Grande',Verdana,Arial,sans-serif;font-size:12px;line-height:16px;color:rgb(17,85,204)">503.707.9018</a><span style="color:rgb(102,102,102);font-family:'Lucida Grande',Verdana,Arial,sans-serif;font-size:12px;line-height:16px"> // Twitter: iovation</span><br>
<strong style="color:rgb(0,102,51);font-family:'Lucida Grande',Verdana,Arial,sans-serif;font-size:12px;line-height:16px"><a href="http://www.iovation.com/" title="Visit iovation.com" style="color:rgb(0,102,51);text-decoration:none;border-bottom-width:0px;border-bottom-style:initial;border-bottom-color:initial" target="_blank">www.iovation.com</a></strong><br>
<font color="#006633" face="Lucida Grande, Verdana, Arial, sans-serif"><span style="font-size:12px;line-height:15px"><br></span></font><span><font color="#888888"><div><font color="#006633" face="'Lucida Grande', Verdana, Arial, sans-serif"><span style="font-size:12px;line-height:16px"><img src="http://www.iovation.com/images/signature/eml_sig_mobilefraud.gif"></span></font></div>
</font></span></div>
<br><br><div class="gmail_quote">On Sat, Oct 12, 2013 at 10:04 AM, Rowan Thorpe <span dir="ltr"><<a href="mailto:rowan@rowanthorpe.com" target="_blank">rowan@rowanthorpe.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Quick Summary for the tl;dr people:<br>
<br>
(A) Follow the IXP-M wiki exclusively if you have lots of free time,<br>
and don't mind rediscovering platform quirks and having to recreate<br>
your workarounds each time, and don't mind the risk of accidentally<br>
re-creating settings incorrectly due to human-error each time<br>
<br>
(B) Use it in parallel with reading the commands in the auto-installer<br>
script if you wish to still be bullet-proof but want to save time (on<br>
Debian, thus far) or if you can't think up workarounds that work for<br>
you<br>
<br>
(C) Only actually run the autoinstaller (on a bare or fully backed up<br>
OS!) if you want to automate everything and you feel reckless, and don't<br>
mind hacking it a bit to fit with latest IXP-M versions<br>
<br>
====<br>
<br>
Long version:<br>
<br>
It seems that what my script actually does (and what it doesn't do)<br>
has been a bit misunderstood...<br>
<div class="im"><br>
> Probably best to stick with the standard installation process and<br>
> restore the mysql dump.<br>
<br>
</div>I agree with this entirely, which doesn't conflict with also referring<br>
to the auto-installer for tips. The script was designed to follow "the<br>
standard installation process" with three extra vital distinctions:<br>
<br>
(1) automated (after editing the simple installer-conf once)<br>
(2) as forward-compatible as possible with future IXP-M versions<br>
(3) easily update-able to handle various platforms' quirks and<br>
differences<br>
<br>
The script actually does restore the mysql dump, and the other steps<br>
Brian mentioned in his first email, and in fact it automates "the<br>
standard installation process" to the letter (as it appeared in the<br>
IXP-M wiki on 29/09/2013), but it also takes (optional) extra steps to<br>
attempt to have a fully functional, populated and running installation:<br>
<br>
(A) based on a potentially later version of IXP-M than was previously<br>
used<br>
(B) with configurable locations e.g. log files in /var/log/ixpmanager,<br>
cache files in /var/cache/ixpmanager (allowing a savvy sysadmin to<br>
gain security and performance benefits, etc)<br>
(C) even on a new server, without having to resort to<br>
fragile/maintenance-hell commands like<br>
"cd / && tar -x -f THE_PREVIOUS_FILESYSTEM_FOOTPRINT.tar" in order<br>
to catch subtle changes to locations like /usr/local/ixp,<br>
/opt/ixpmanager, /usr/local/share/perl/5.x.x/IXP-Manager,<br>
/etc/ixpmanager.conf, /var/cache/ixpmanager, /var/log/ixpmanager,<br>
etc...<br>
(D) by updating (rather than overwriting) the config from a<br>
fresh OS install on the fly, based on its simple key=value format<br>
config file, so that newer versions of IXP-M can hopefully still<br>
work without having to manually investigate and edit the entire<br>
application.ini.dist each time to discover additions,<br>
modifications and deletions that would be blitzed by just copying<br>
the old configs into place<br>
(E) by integrating things like sflow, smokeping, mrtg, etc without<br>
having to manually re-edit or copy-paste files at various points in<br>
the filesystem<br>
(F) by copying extra things like skinned templates and image files into<br>
place<br>
<br>
Copy-pasted below is taken from<br>
<a href="https://github.com/rowanthorpe/ixp-autoinstall/blob/master/ixp-autoinstall.sh#L157" target="_blank">https://github.com/rowanthorpe/ixp-autoinstall/blob/master/ixp-autoinstall.sh#L157</a><br>
and shows the steps it takes, which all default to "on" (=1), and can<br>
all be independently turned off with flags (e.g. if you have already<br>
manually edited or copied the config and trust it to be up-to-date, or<br>
if you have already installed a system package for sflowtool, etc). If<br>
you disable enough of the extra steps you will end up with exactly the<br>
"standard installation process":<br>
<br>
====<br>
do_install_firewall=1<br>
do_install_deps=1<br>
do_setup_dirs=1<br>
do_setup_permissions=1<br>
do_install_ixpmanager=1<br>
do_create_database=1<br>
do_edit_base_confs=1<br>
do_setup_schema=1<br>
do_setup_webserver=1<br>
do_setup_fixtures=1<br>
do_unarchive_skin=1<br>
do_unarchive_misc=1<br>
do_unarchive_ext_images=1<br>
do_setup_maintenance_file=1<br>
do_populate_db_data=1<br>
do_setup_perl_libs=1<br>
do_integrate_mrtg=1<br>
do_setup_mrtg=1<br>
do_setup_mrtg_init=1<br>
do_setup_periodic_update=1<br>
do_setup_periodic_update_cron=1<br>
do_setup_store_traffic_cron=1<br>
do_setup_update_macs=1<br>
do_setup_poll_switch_cron=1<br>
do_install_sflowtool=1<br>
do_setup_rrdcached=1<br>
do_setup_sflow_to_rrd=1<br>
do_setup_sflow_to_rrd_init=1<br>
do_let_sflow_through_firewall=1<br>
do_integrate_sflow=1<br>
do_setup_smokeping=1<br>
do_integrate_smokeping=1<br>
do_remove_build_deps=1<br>
do_start_webserver=1<br>
do_let_web_through_firewall=1<br>
====<br>
<br>
I hoped that people would like to have that script around, even if only<br>
as a quick "to do" reference regarding streamlining the installation<br>
process (and showing what is not yet pointed out in the docs), but I've<br>
not received any feedback or help with updating it yet (or integrating<br>
any of its workarounds into IXP-M), so unfortunately it has already<br>
fallen a few minor-versions behind IXP-M's master branch because I<br>
don't have time to keep it synchronised by myself, so especially now it<br>
should only be used as a reference rather than actually running it.<br>
However it does serve as a good list of "gotcha's". A funny thing was<br>
that a large part of it was originally dealing with hacks to make all<br>
the mrtg perl-fu work correctly behind the scenes, and as I was about to<br>
upload it IXP-M was updated to internalise much of that with PHP, so<br>
that part became redundant. I believe that most of the rest of the<br>
auto-installer can eventually be made redundant in the same fashion,<br>
and if people check it out that may happen sooner...<br>
<div class="im"><br>
> > Plugging away at your autoinstall script. Most of it is tweaking<br>
> > the conf.sh file.<br>
<br>
</div>If you already have a working config *for the present master HEAD<br>
version of IXP-M* then it will be easier to copy that file and just<br>
skip that step (as in what "--no-edit-base-confs" does)<br>
<div class="im"><br>
> > I have found that it would be useful to include at least in the<br>
> > documentation of all the package dependencies the script requires.<br>
<br>
</div>Of course those deps are all inside "case" statements because those<br>
are the deps specifically needed in Debian Wheezy (and therefore<br>
Ubuntu, etc). For example several platforms already have an sflowtool<br>
package, but there is not one in Debian yet. However, I have packaged<br>
and submitted it and am waiting for a Debian Developer to sponsor it<br>
(when that happens sflowtool can become merely a dependency for Debian,<br>
Ubuntu, etc too, rather than requiring the manual compilation step).<br>
<br>
> > ..[snip]..<br>
<div class="im">> > 10 apt-get install memcached<br>
> > 11 apt-get install subversion<br>
> > 12 apt-get install php5-memcache<br>
> > 13 apt-get install php5-snmp<br>
> > 14 apt-get install php-apc<br>
</div>> > ..[snip]..<br>
<div class="im">> > Found pear was missing<br>
> ><br>
> > 26 pear channel-discover <a href="http://pear.symfony.com" target="_blank">pear.symfony.com</a><br>
</div><div class="im">> > 27 apt-get install php-pear<br>
> > 28 pear channel-discover <a href="http://pear.symfony.com" target="_blank">pear.symfony.com</a><br>
</div><div class="im">> > 29 pear channel-discover <a href="http://pear.doctrine-project.org" target="_blank">pear.doctrine-project.org</a><br>
> > 30 pear install doctrine/DoctrineORM<br>
<br>
</div>A list of Debian Wheezy deps, separated into those with<br>
version-requirements, interactive or not, etc, are at<br>
<a href="https://github.com/rowanthorpe/ixp-autoinstall/blob/master/ixp-autoinstall.sh#L595" target="_blank">https://github.com/rowanthorpe/ixp-autoinstall/blob/master/ixp-autoinstall.sh#L595</a><br>
This should fall more-or-less in-line with what you need on Ubuntu.<br>
<br>
> > ..[snip]..<br>
<div class="im">> > Then found I needed to flip a switch in the install.sh for mysql<br>
> > passwords.<br>
<br>
</div>The script processes the $with_root_db_password from the autoinstall<br>
conf for this.<br>
<br>
> > ..[snip]..<br>
> > Discovered MRTG not installed.<br>
<br>
It is included as a dep here:<br>
<a href="https://github.com/rowanthorpe/ixp-autoinstall/blob/master/ixp-autoinstall.sh#L621" target="_blank">https://github.com/rowanthorpe/ixp-autoinstall/blob/master/ixp-autoinstall.sh#L621</a><br>
<br>
> > ..[snip]..<br>
<div class="im">> > Currently I am still on MRTG with this error.<br>
> ><br>
> > ixp-autoinstall.sh: setting up mrtg<br>
> > sed: can't read /etc/mrtg-rrd.conf: No such file or directory<br>
> ><br>
> > I haven't been able to determine where in the recipe this file gets<br>
> > built.<br>
<br>
</div>mrtg-rrd is included as a dep here:<br>
<a href="https://github.com/rowanthorpe/ixp-autoinstall/blob/master/ixp-autoinstall.sh#L622" target="_blank">https://github.com/rowanthorpe/ixp-autoinstall/blob/master/ixp-autoinstall.sh#L622</a><br>
I don't remember exactly any more but I think I included mrtg-rrd<br>
because those who want to can push their mrtg data to rrd too using<br>
this. The installer would bork at that point because the config file<br>
won't exist if you didn't install the package. It is just trying to<br>
update it to point to where IXP-manager has put its own mrtg config. I<br>
am pretty sure it is an "optional extra" though. If in doubt ignore that<br>
command (and that package).<br>
<div class="im"><br>
> > > Setting variables is one thing, Understanding the whole script<br>
> > > might be another.<br>
<br>
</div>The "just setting variables" approach is when running the autoinstaller<br>
with all steps enabled, having edited its config, but as mentioned<br>
before it has already fallen slightly out of synch with the latest<br>
IXP-M so you shouldn't rely on that approach any more. The way to use<br>
it for now should be by reading through the script (just copying your<br>
configs and ignoring all the sed-code for that if it makes you<br>
cross-eyed) and manually entering its commands which either (1)<br>
coincide with the latest version of the IXP-M wiki, or (2) take<br>
steps which are missing from the wiki *and which still make sense with<br>
the latest IXP-M version*.<br>
<br>
The fully-automated aspect of the script attempts to cater to those who<br>
want to effectively "provision" IXP-Manager to non-interactively<br>
migrate VMs, etc.<br>
<div class="im"><br>
--<br>
Rowan Thorpe<br>
mailto:<a href="mailto:rowan@rowanthorpe.com">rowan@rowanthorpe.com</a><br>
</div><div class="im">PGP fingerprint:<br>
BB0A 0787 C0EE BDD8 7F97 3D30 49F2 13A5 265D CCBD<br>
----<br>
"There is a great difference between worry and concern. A worried<br>
person sees a problem, and a concerned person solves a problem."<br>
- Harold Stephens<br>
_______________________________________________<br>
INEX IXP Manager mailing list<br>
<a href="mailto:ixpmanager@inex.ie">ixpmanager@inex.ie</a><br>
</div><a href="https://www.inex.ie/mailman/listinfo/ixpmanager" target="_blank">https://www.inex.ie/mailman/listinfo/ixpmanager</a><br>
</blockquote></div><br></div>