Fast dependency installation (gem, brew, pip, npm)
At Wikia we write a lot of scripts and use a lot of tools to manage our App Farm. Anyone in the team should be able to use the scripts in the same way, so we created a little script that installs all the dependencies.
This file initially looked like that:
#!/bin/bash sudo easy_install pip sudo pip install invoke sudo pip install termcolor [..] sudo gem install cocoapods -V sudo gem install shenzhen -V [...] ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" brew update brew install git brew install wget [...] sudo npm install chromedriver -g sudo npm install ios-sim -g [...]
This worked initially, but as the file grew, we had to rerun the script on every computer to download the missing dependencies and then the problems started to rise.
- Reinstalling of things you already have is taking a long time
npmreinstall all the packages by default. They don't check if packages are installed already
- If something failed to install, the information got lost in all the logs
So the script was rewritten in Python and learned some new tricks:
check_call(['which pip || sudo easy_install pip'], shell=True) check_call(['which brew || ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"'], shell=True)
For commands installing binaries we can check if
which has some output, meaning this binary is already installed.
def pip(package, pip_list=check_output(['pip', 'list'])): print "pip " + package package_with_space = package + " " if package_with_space not in pip_list: cmd = ['sudo', 'pip', 'install', package, '--upgrade'] check_call(cmd)
Installing pip packages can use
pip list and first check if the packages are installed. Using
pip list as a default argument we only run it once and then check against it in subsequent calls.
def gem(package, gem_list=check_output(['gem', 'list'])): print "gem " + package package_with_space = package + " " if package_with_space not in gem_list: cmd = ['sudo', 'gem', 'install', package, '-V', '--no-ri', '--no-rdoc'] check_call(cmd)
The same idea as with
pip, but we're using
gem list. We also add
--no-rdoc as we don't care about documentation and such.
def brew(package): print "brew " + package cmd = ['brew', 'install', package] try: check_call(cmd) except CalledProcessError: pass
Brew checks if the package is already installed, but sometimes the error codes are inconsistent. That's why we
try: except: and ignore the result.
def npm(package): print "npm " + package try: check_call(['which', package]) except CalledProcessError: cmd = ['sudo', 'npm', 'install', package, '-g'] check_call(cmd)
We only use
And the rest of the file looks pretty similar:
pip('invoke') pip('termcolor') [...] gem('cocoapods') gem('shenzhen') [...] brew('git') brew('wget') [...] npm('chromedriver') npm('ios-sim') [...]
With this solution we only check what's needed and install that. We crash as early as possible to debug some installation.
The only problem is that we don't check version numbers, but we haven't had any problems with that yet. If any problem occurs we could add version checking to the script.