[ixpmanager] Server rebuild
Brian Thompson
brian.thompson at iovation.com
Sat Oct 12 18:33:35 IST 2013
Rowan,
Thank you so much. I had ran out of time, and that was just the last bump
in the road I had hit.
I was able to spend more time on this, and reading your script. I was able
to glean that I was not using "Debian/Wezzy" so it was not checking all of
the depencies.
I have ran them manually and am getting much further. I think I am almost
all of the way through. I will have to spend more time on this later this
weekend.
Cheers,
Brian
*Brian Thompson*
Senior Infrastructure Engineer // Senior Second Guesser
Direct: 503.943.6779
Mobile: 503.707.9018 // Twitter: iovation
*www.iovation.com*
On Sat, Oct 12, 2013 at 10:04 AM, Rowan Thorpe <rowan at rowanthorpe.com>wrote:
> Quick Summary for the tl;dr people:
>
> (A) Follow the IXP-M wiki exclusively if you have lots of free time,
> and don't mind rediscovering platform quirks and having to recreate
> your workarounds each time, and don't mind the risk of accidentally
> re-creating settings incorrectly due to human-error each time
>
> (B) Use it in parallel with reading the commands in the auto-installer
> script if you wish to still be bullet-proof but want to save time (on
> Debian, thus far) or if you can't think up workarounds that work for
> you
>
> (C) Only actually run the autoinstaller (on a bare or fully backed up
> OS!) if you want to automate everything and you feel reckless, and don't
> mind hacking it a bit to fit with latest IXP-M versions
>
> ====
>
> Long version:
>
> It seems that what my script actually does (and what it doesn't do)
> has been a bit misunderstood...
>
> > Probably best to stick with the standard installation process and
> > restore the mysql dump.
>
> I agree with this entirely, which doesn't conflict with also referring
> to the auto-installer for tips. The script was designed to follow "the
> standard installation process" with three extra vital distinctions:
>
> (1) automated (after editing the simple installer-conf once)
> (2) as forward-compatible as possible with future IXP-M versions
> (3) easily update-able to handle various platforms' quirks and
> differences
>
> The script actually does restore the mysql dump, and the other steps
> Brian mentioned in his first email, and in fact it automates "the
> standard installation process" to the letter (as it appeared in the
> IXP-M wiki on 29/09/2013), but it also takes (optional) extra steps to
> attempt to have a fully functional, populated and running installation:
>
> (A) based on a potentially later version of IXP-M than was previously
> used
> (B) with configurable locations e.g. log files in /var/log/ixpmanager,
> cache files in /var/cache/ixpmanager (allowing a savvy sysadmin to
> gain security and performance benefits, etc)
> (C) even on a new server, without having to resort to
> fragile/maintenance-hell commands like
> "cd / && tar -x -f THE_PREVIOUS_FILESYSTEM_FOOTPRINT.tar" in order
> to catch subtle changes to locations like /usr/local/ixp,
> /opt/ixpmanager, /usr/local/share/perl/5.x.x/IXP-Manager,
> /etc/ixpmanager.conf, /var/cache/ixpmanager, /var/log/ixpmanager,
> etc...
> (D) by updating (rather than overwriting) the config from a
> fresh OS install on the fly, based on its simple key=value format
> config file, so that newer versions of IXP-M can hopefully still
> work without having to manually investigate and edit the entire
> application.ini.dist each time to discover additions,
> modifications and deletions that would be blitzed by just copying
> the old configs into place
> (E) by integrating things like sflow, smokeping, mrtg, etc without
> having to manually re-edit or copy-paste files at various points in
> the filesystem
> (F) by copying extra things like skinned templates and image files into
> place
>
> Copy-pasted below is taken from
>
> https://github.com/rowanthorpe/ixp-autoinstall/blob/master/ixp-autoinstall.sh#L157
> and shows the steps it takes, which all default to "on" (=1), and can
> all be independently turned off with flags (e.g. if you have already
> manually edited or copied the config and trust it to be up-to-date, or
> if you have already installed a system package for sflowtool, etc). If
> you disable enough of the extra steps you will end up with exactly the
> "standard installation process":
>
> ====
> do_install_firewall=1
> do_install_deps=1
> do_setup_dirs=1
> do_setup_permissions=1
> do_install_ixpmanager=1
> do_create_database=1
> do_edit_base_confs=1
> do_setup_schema=1
> do_setup_webserver=1
> do_setup_fixtures=1
> do_unarchive_skin=1
> do_unarchive_misc=1
> do_unarchive_ext_images=1
> do_setup_maintenance_file=1
> do_populate_db_data=1
> do_setup_perl_libs=1
> do_integrate_mrtg=1
> do_setup_mrtg=1
> do_setup_mrtg_init=1
> do_setup_periodic_update=1
> do_setup_periodic_update_cron=1
> do_setup_store_traffic_cron=1
> do_setup_update_macs=1
> do_setup_poll_switch_cron=1
> do_install_sflowtool=1
> do_setup_rrdcached=1
> do_setup_sflow_to_rrd=1
> do_setup_sflow_to_rrd_init=1
> do_let_sflow_through_firewall=1
> do_integrate_sflow=1
> do_setup_smokeping=1
> do_integrate_smokeping=1
> do_remove_build_deps=1
> do_start_webserver=1
> do_let_web_through_firewall=1
> ====
>
> I hoped that people would like to have that script around, even if only
> as a quick "to do" reference regarding streamlining the installation
> process (and showing what is not yet pointed out in the docs), but I've
> not received any feedback or help with updating it yet (or integrating
> any of its workarounds into IXP-M), so unfortunately it has already
> fallen a few minor-versions behind IXP-M's master branch because I
> don't have time to keep it synchronised by myself, so especially now it
> should only be used as a reference rather than actually running it.
> However it does serve as a good list of "gotcha's". A funny thing was
> that a large part of it was originally dealing with hacks to make all
> the mrtg perl-fu work correctly behind the scenes, and as I was about to
> upload it IXP-M was updated to internalise much of that with PHP, so
> that part became redundant. I believe that most of the rest of the
> auto-installer can eventually be made redundant in the same fashion,
> and if people check it out that may happen sooner...
>
> > > Plugging away at your autoinstall script. Most of it is tweaking
> > > the conf.sh file.
>
> If you already have a working config *for the present master HEAD
> version of IXP-M* then it will be easier to copy that file and just
> skip that step (as in what "--no-edit-base-confs" does)
>
> > > I have found that it would be useful to include at least in the
> > > documentation of all the package dependencies the script requires.
>
> Of course those deps are all inside "case" statements because those
> are the deps specifically needed in Debian Wheezy (and therefore
> Ubuntu, etc). For example several platforms already have an sflowtool
> package, but there is not one in Debian yet. However, I have packaged
> and submitted it and am waiting for a Debian Developer to sponsor it
> (when that happens sflowtool can become merely a dependency for Debian,
> Ubuntu, etc too, rather than requiring the manual compilation step).
>
> > > ..[snip]..
> > > 10 apt-get install memcached
> > > 11 apt-get install subversion
> > > 12 apt-get install php5-memcache
> > > 13 apt-get install php5-snmp
> > > 14 apt-get install php-apc
> > > ..[snip]..
> > > Found pear was missing
> > >
> > > 26 pear channel-discover pear.symfony.com
> > > 27 apt-get install php-pear
> > > 28 pear channel-discover pear.symfony.com
> > > 29 pear channel-discover pear.doctrine-project.org
> > > 30 pear install doctrine/DoctrineORM
>
> A list of Debian Wheezy deps, separated into those with
> version-requirements, interactive or not, etc, are at
>
> https://github.com/rowanthorpe/ixp-autoinstall/blob/master/ixp-autoinstall.sh#L595
> This should fall more-or-less in-line with what you need on Ubuntu.
>
> > > ..[snip]..
> > > Then found I needed to flip a switch in the install.sh for mysql
> > > passwords.
>
> The script processes the $with_root_db_password from the autoinstall
> conf for this.
>
> > > ..[snip]..
> > > Discovered MRTG not installed.
>
> It is included as a dep here:
>
> https://github.com/rowanthorpe/ixp-autoinstall/blob/master/ixp-autoinstall.sh#L621
>
> > > ..[snip]..
> > > Currently I am still on MRTG with this error.
> > >
> > > ixp-autoinstall.sh: setting up mrtg
> > > sed: can't read /etc/mrtg-rrd.conf: No such file or directory
> > >
> > > I haven't been able to determine where in the recipe this file gets
> > > built.
>
> mrtg-rrd is included as a dep here:
>
> https://github.com/rowanthorpe/ixp-autoinstall/blob/master/ixp-autoinstall.sh#L622
> I don't remember exactly any more but I think I included mrtg-rrd
> because those who want to can push their mrtg data to rrd too using
> this. The installer would bork at that point because the config file
> won't exist if you didn't install the package. It is just trying to
> update it to point to where IXP-manager has put its own mrtg config. I
> am pretty sure it is an "optional extra" though. If in doubt ignore that
> command (and that package).
>
> > > > Setting variables is one thing, Understanding the whole script
> > > > might be another.
>
> The "just setting variables" approach is when running the autoinstaller
> with all steps enabled, having edited its config, but as mentioned
> before it has already fallen slightly out of synch with the latest
> IXP-M so you shouldn't rely on that approach any more. The way to use
> it for now should be by reading through the script (just copying your
> configs and ignoring all the sed-code for that if it makes you
> cross-eyed) and manually entering its commands which either (1)
> coincide with the latest version of the IXP-M wiki, or (2) take
> steps which are missing from the wiki *and which still make sense with
> the latest IXP-M version*.
>
> The fully-automated aspect of the script attempts to cater to those who
> want to effectively "provision" IXP-Manager to non-interactively
> migrate VMs, etc.
>
> --
> Rowan Thorpe
> mailto:rowan at rowanthorpe.com
> PGP fingerprint:
> BB0A 0787 C0EE BDD8 7F97 3D30 49F2 13A5 265D CCBD
> ----
> "There is a great difference between worry and concern. A worried
> person sees a problem, and a concerned person solves a problem."
> - Harold Stephens
> _______________________________________________
> INEX IXP Manager mailing list
> ixpmanager at inex.ie
> https://www.inex.ie/mailman/listinfo/ixpmanager
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.inex.ie/pipermail/ixpmanager/attachments/20131012/d627bc63/attachment.html>
More information about the ixpmanager
mailing list