How to keep a local ~/bin folder up to date with updates from various Github repositories?

Question

~/bin contains a number of scripts that I downloaded from various Github repositories for which the Ubuntu default repositories do not provide the latest version and/or PPAs do not exist.

What are good ways to keep all these scripts automatically up to date and synchronised with the most recent versions from their respective Github repositories?

Answer 1

Simple solution

What you want to do essentially is to run git fetch && git pull from inside each one of the repository folders inside ~/bin. The simplest way is to add this command to Startup Applications:

find /home/user/bin  -type d -exec sh -c 'cd {} && git fetch && git pull' \;  2> /dev/null

Note, that /home/user/bin must be full path to your ~/bin folder.

---

Python script

But if you want a dedicated utility to this task, here's a python script that updates everything in your ~/bin recursively. Usage is simple - run it from terminal as python git_updater.py and it will take care of the rest. That same command can be even added as entry to Startup Applications to run the script each time you log in

The script source is below, and is also available on my GitHub repository.

#!/usr/bin/env python3
# Author: Serg Kolo
# Date: 10/21/2016
# Written for: https://askubuntu.com/q/759058/295286
from __future__ import print_function
import subprocess
import os

def run_cmd(cmdlist,workdir):
     """ utility: reusable function for 
         running external commands """
     new_env = dict(os.environ)
     new_env['LC_ALL'] = 'C'
     try:
         stdout = subprocess.check_output(
                    cmdlist, 
                    env=new_env,
                    cwd=workdir
         )
     except subprocess.CalledProcessError:
         pass
     else:
         if stdout:
             return stdout

def is_behind(cwd):
    """ simple wrapper for checking
        git status output
    """
    fetch = run_cmd(['git','fetch','origin','master'],cwd)
    status = run_cmd(['git','status'],cwd)
    if 'Your branch is behind' in status.decode():
        return True

def update(cwd):
    print('> previous commit:')
    print(run_cmd(['git','--no-pager','log','-1'],cwd).decode())
    print(run_cmd(['git','pull'],cwd).decode())
    print('> latest commit:')
    print(run_cmd(['git','--no-pager','log','-1'],cwd).decode())

def main():
    root_dir = os.path.join(os.path.expanduser('~'),'bin/')
    base_cmd = ['git','--no-pager']
    first_args = ['status']
    second_args = ['log','-1']
    third_args = ['pull']

    for root,dirs,files in os.walk(root_dir):
        for dir in dirs:
            top_dir = os.path.join(root,dir)
            git_dir = os.path.join(top_dir,'.git')
            if os.path.exists(git_dir):
                print('WORKING REPOSITORY:' + top_dir)
                if is_behind(top_dir):
                    print('repository is behind, will update')
                    update(top_dir)
                else:
                    print('not behind, skipped')
                print('-'*80)

if __name__ == '__main__': main()

---

Script Demo

$ python git_updater.py                                                       
WORKING REPOSITORY:/home/xieerqi/bin/sergrep
From https://github.com/SergKolo/sergrep
 * branch            master     -> FETCH_HEAD
repository is behind, will update
> previous commit:
commit ba629ef71c4d2bfe329b8ebe5f48d1ce21e76ebc
Author: SergKolo <1047481448@qq.com>
Date:   Wed Aug 24 21:22:06 2016 -0600

    fixed sending sigterm to process

Updating ba629ef..dd0c962
Fast-forward
 disable_super_key.py   |   2 +-
 git_updater.py         |  61 ++++++++++++
 launcherctl.py         | 162 +++++++++++++++++++++++++++++++
 single_window_expo.py  | 138 ++++++++++++++++++++++++++
 windowctrl.py          | 258 +++++++++++++++++++++++++++++++++++++++++++++++++
 xml_wallpaper_maker.py | 152 +++++++++++++++++++++++++++++
 6 files changed, 772 insertions(+), 1 deletion(-)
 create mode 100644 git_updater.py
 create mode 100755 launcherctl.py
 create mode 100755 single_window_expo.py
 create mode 100755 windowctrl.py
 create mode 100755 xml_wallpaper_maker.py

> latest commit:
commit dd0c9627392540df7c4230e7898cb9e55adc7083
Author: SergKolo <1047481448@qq.com>
Date:   Fri Oct 21 02:37:01 2016 -0600

    git updater script

--------------------------------------------------------------------------------
WORKING REPOSITORY:/home/xieerqi/bin/PythonStuff
From https://github.com/mnovits1/PythonStuff
 * branch            master     -> FETCH_HEAD
not behind, skipped
--------------------------------------------------------------------------------