Leo on Tech

Blog

  • Unix command of the day: watch

    The project I’m working on right now involve not just Dockerized Rails microservices, Meteor JS, and a data set measured in tens of terabytes, but also a big Bash code base.

    Bash is a language that makes it easy to shoot yourself in the foot. I have some thoughts on how to write robust, modular, loosely coupled, unit tested Bash that go beyond Bash strict mode and shellcheck. However, let’s save those for later.

    Here’s a useful shell command for today: watch

    watch will run a command for you every few seconds and output the results on a clean screen. If you combine it with a split-screen or split-pane tool, you can quickly create a mini-dashboard.

    For example, watch -n15 df -h will print your free disk space every 15 seconds:

    Every 15.0s: df -h        Tue Apr 14 13:43:25 2015

Filesystem            Size  Used Avail Use% Mounted on
/dev/sda3             1.8T  691G  1.1T  40% /
tmpfs                 253G   12K  253G   1% /dev/shm
/dev/sda1             248M   58M  178M  25% /boot
/dev/sdb1              30T   12T   18T  41% /my_disk

    By way of another example, watch -n60 db2 list utilities show detail will check on the status of your DB2 load and other operations every 60 seconds.

    Every 60.0s: db2 list utilities show detail            Mon Apr 13 12:06:24 2015

ID                               = 4
Type                             = LOAD
Database Name                    = MY_DB
Member Number                    = 0
Description                      = [LOADID: 1177.2015-04-13-15.58.12.282010.0 (20;15)] 
[*LOCAL.MY_USER.150413204511] OFFLINE LOAD DEL
AUTOMATIC INDEXING INSERT NON-RECOVERABLE MY_SCHEMA    .MY_TABLE
Start Time                       = 04/13/2015 15:58:12.298788
State                            = Executing
Invocation Type                  = User
Progress Monitoring:
   Phase Number                  = 1
      Description                = SETUP
      Total Work                 = 0 bytes
      Completed Work             = 0 bytes
      Start Time                 = 04/13/2015 15:58:12.298799

   Phase Number [Current]        = 2
      Description                = LOAD
      Total Work                 = 4105362974 rows
      Completed Work             = 1069904365 rows
      Start Time                 = 04/13/2015 15:58:15.821695

   Phase Number                  = 3
      Description                = BUILD
      Total Work                 = 2 indexes
      Completed Work             = 0 indexes
      Start Time                 = Not Started

    Mac OS X does not include watch by default. Assuming you have Homebrew, the following will install watch for you:

    brew install watch

  • libdb2.so.1: cannot open shared object file

    I got this error starting a ruby application:

    /usr/local/rvm/gems/ruby-1.9.3-p547/gems/bundler-1.7.3/lib/bundler/runtime.rb:76:in `require': 
libdb2.so.1: cannot open shared object file: 
No such file or directory - 
/usr/local/rvm/gems/ruby-1.9.3-p547/extensions/x86_64-linux/1.9.1/ibm_db-2.5.11/ibm_db.so (LoadError)

    An .so is a Linux library, equivalent to a .dll on Windows or a .dylib on Mac. Note that there are two different libraries mentioned. ibm_db.so is present, while libdb2.so.1 is missing.

    You can verify the dependencies using the ldd command:

    $ ldd /usr/local/rvm/gems/ruby-1.9.3-p547/extensions/x86_64-linux/1.9.1/ibm_db-2.5.11/ibm_db.so
    linux-vdso.so.1 =>  (0x00007fff93545000)
    libruby.so.1.9 => /usr/local/rvm/rubies/ruby-1.9.3-p547/lib/libruby.so.1.9 (0x00007fb8ba5a5000)
    libdb2.so.1 => not found
    libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fb8ba37f000)
    librt.so.1 => /lib64/librt.so.1 (0x00007fb8ba177000)
    libdl.so.2 => /lib64/libdl.so.2 (0x00007fb8b9f72000)
    libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007fb8b9d3b000)
    libm.so.6 => /lib64/libm.so.6 (0x00007fb8b9ab7000)
    libc.so.6 => /lib64/libc.so.6 (0x00007fb8b9722000)
    /lib64/ld-linux-x86-64.so.2 (0x0000003730400000)
    libfreebl3.so => /lib64/libfreebl3.so (0x00007fb8b94ab000)

    For me, the issue was caused by a missing IBM Data Server Driver directory. IBM_DB_LIB was pointing to a non-existent directory:

    $ set | grep IBM
IBM_DB_DIR=/tmp/dsdriver/.
IBM_DB_HOME=/tmp/dsdriver/.
IBM_DB_INCLUDE=/tmp/dsdriver/./include
IBM_DB_LIB=/tmp/dsdriver/./lib

    Reinstalling the Data Server Driver restored the libdb2.so.1 and eliminated the error.

    (If you run into this issue with different libraries, you will likely need to examine your LD_LIBRARY_PATH environment variable and use the ldconfig command to reload any changes.)

  • How to run and install IPMIView on Mac

    Edit: The download link doesn’t work anymore, but it looks like IPMIView is now available in the Mac App Store. Also, just in case, someone suggested running the Linux version on Mac.

    Softlayer cloud uses IPMIView for direct console access to bare metal hardware. SuperMicro makes a Mac version of IPMIView available for download.

    Bizarrely, SuperMicro doesn’t appear to have ever tested it, because double-clicking on the downloaded IPMIView20 application doesn’t do anything. This is because someone forgot to set the execute permission on the installer.

    Supposing it’s in your downloads folder, open up Terminal and run the following commands:

    cd /Users/$( whoami )/Downloads/IPMIView20.app/Contents/MacOS
chmod u+x IPMIView20
./IPMIView20

    This will open up the installer for you. Once it’s installed, it will show up in your Launchpad like a normal application.

    Launchpad

    Save

    Save

  • Persistent SSH sessions with screen

    Do you ever need to kick off a long-running command while SSHed to a server, but be able to disconnect and reconnect at will? You can do this with screen.

    Before doing anything, start a screen session:

    screen

    When you’re ready to put your work on hold, detach the screen:

    screen -d

    If you have a long-running command running, you can detach that screen from a different shell session by specifying the process id:

    # look up the process id
ps aux | grep SCREEN
# detach that screen
screen -d <my_process_id>

    You can now disconnect from the server safely.

    When you reconnect, you can also reconnect to your screen session:

    screen -r

     

  • How to rename a file in a File upload dialog on Mac

    Windows users will scoff at this, but renaming a file in a File Upload dialog box on Mac is a surprisingly obscure action. Renaming is not available in the context menu, nor does the usual shortcut work.

    Screenshot 2014-03-24 18.01.26To rename a file in a regular Finder window, you can select it and hit the Enter key. However, you can’t do this in a File Upload dialog as the Enter key has a different meaning.

    To rename a file, select it, hit Cmd+I, open up the Name & Extension pane, and change the name.

  • Windows to Mac: Keyboard shortcuts

    In December, I got a Macbook Air that I’m now using as a primary development machine. Before that, I was developing on a Windows 7 machine with heavy reliance on Git bash and Cygwin, and using Linux on the server. I’ve used Linux as a primary desktop at times, but found both dual-boot and VMs too much of a hassle.

    I’m enjoying the Mac, but one of the biggest adjustments is keyboard shortcuts.

    Here are some shortcuts that are specific to the Mac Terminal (and Terminal alternatives like iTerm 2):

    [vtftable ]
    Mac ;;; Mac Terminal ;;; Windows ;;; Action ;nn;
    Cmd+Left Arrow ;;; Ctrl+A ;;; Home ;;; Go to start of line ;nn;
    Cmd+Right Arrow ;;; Ctrl+E ;;; End ;;; Go to end of line ;nn;
    Shift+Cmd+Left Arrow ;;; ;;; Shift+Home ;;; Select to start of line ;nn;
    Shift+Cmd+Right Arrow ;;; ;;; Shift+End ;;; Select to end of line ;nn;
    ;;; Ctrl+K ;;; ;;; Clear line after cursor ;nn;
    ;;; Ctrl+U ;;; ;;; Clear line before cursor;nn;
    [/vtftable]

    Some general shortcuts:

    [vtftable ]
    Mac ;;; Windows ;;; Action ;nn;
    Cmd+Down Arrow;;; PageDn ;;; Go one page down ;nn;
    Cmd+Up Arrow ;;; PageUp ;;; Go one page up ;nn;
    ;;; ;;; ;nn;
    Cmd+Space ;;; Windows key ;;; Start menu/Spotlight ;nn;
    Cmd+Opt+Esc ;;; Ctrl+Shift+Esc ;;; Open Task Manager ;nn;
    Cmd+Delete ;;; Delete ;;; Delete selected files ;nn;
    ;;; ;;; ;nn;
    Cmd+Delete ;;; Delete ;;; Delete character in front of the cursor ;nn;
    ;;; ;;; ;nn;
    Cmd+R ;;; F5 ;;; Reload current webpage ;nn;
    ;;; ;;; ;nn;
    Cmd+Shift+3 ;;; ;;; Save screenshot to desktop/Dropbox ;nn;
    Cmd+Ctrl+Shift+3 ;;; PrtScr ;;; Copy screenshot to clipboard ;nn;
    Cmd+Shift+4 ;;; ;;; Save screenshot of an area of screen to desktop/Dropbox ;nn;
    Cmd+Ctrl+Shift+4 ;;; PrtScr (SnagIt) ;;; Copy screenshot of an area of screen to clipboard ;nn;
    [/vtftable]

    Of course, there are many more shortcuts.

  • libdb2.so.1: cannot open shared object file: No such file or directory – … ibm_db.so

    Got this error while deploying a Rails app on Nginx:

    libdb2.so.1: cannot open shared object file: No such file or directory - ... ibm_db.so

    This means that the ibm_db adapter is installed, but it can’t find the DB2 libraries. The issue is that IBM_DB_HOME and some other environment variables are not set.

    The best solution is to make sure all users have db2profile loaded. Edit /etc/profile and add:

    . /opt/dsdriver/db2profile

    You should now reload your profile (. /etc/profile) and restart Nginx.

    This assumes that you already have IBM Data Server Driver installed under /opt/dsdriver.

  • Linux command of the day: banner

    banner can be a useful command for setting login and welcome messages (e.g via /etc/profile).

    $ banner PRODUCTION
######  ######  ####### ######  #     #  #####  #######   ###   ####### #     #
#     # #     # #     # #     # #     # #     #    #       #    #     # ##    #
#     # #     # #     # #     # #     # #          #       #    #     # # #   #
######  ######  #     # #     # #     # #          #       #    #     # #  #  #
#       #   #   #     # #     # #     # #          #       #    #     # #   # #
#       #    #  #     # #     # #     # #     #    #       #    #     # #    ##
#       #     # ####### ######   #####   #####     #      ###   ####### #     #

    The Mac implementation is less useful than the GNU/Linux one.

    $ banner -w 40 PRO
          #                         #
          #                         #
          ###########################
          ###########################
          #              #          #
          #              #          #
                         #          #
                         #          #
                         ##        ##
                         ##       ###
                          ##########
                           ########
                             ####
          #                         #
          #                         #
          ###########################
          ###########################
          #             ##          #
          #          #####          #
                  ########          #
               ######### #          #
          # #########    ##        ##
          #########      ##       ###
          #####           ##########
          ###              ########
          #                  ####
          #
                    #######
                ###############
              ###################
            ######          #######
          ####                   ###
          ##                       ##
          #                         #
          #                         #
          #                        ##
          ###                     ###
           ######             #####
             #####################
               #################
                  ###########

     

  • JRuby for the Java .class is .java_class

    I’ve been having a lot of fun working with a Apache jclouds in JRuby. All the examples for the API are in Java and Clojure, while online JRuby docs could be better, so there’ve been some interesting translation challenges.

    I just had to re-Google what .class becomes in JRuby, so a quick note for the future.

    Java:

    template.getOptions().as(EC2TemplateOptions.class).keyPair("bluforcloud-test");

    JRuby:

    template.getOptions.as(EC2TemplateOptions.java_class).keyPair "bluforcloud-test"

     

  • Exploring Ruby and Python interactively

    Both Ruby and Python offer great interactive shells, also known as REPLs (Read Eval Print Loops). These are handy for verifying snippets of code. You can invoke Python’s by simply running python or Ruby’s by running irb, jirb (for jRuby), or rails c (for Rails).

    Sometimes, however, one can be mystified as to what one can do with an object or module. Lately, I’ve been finding the Ruby API documentation especially frustrating.

    Fortunately, both Python and Ruby let you see what’s available. In Python, you can call the dir() function, while Ruby has the handy .methods() method.

    Python:

    >>> dir("some string")
['__add__', '__class__', '__contains__', '__delattr__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__getnewargs__', '__getslice__', '__gt__', '__hash__', '__init__', '__le__', '__len__', '__lt__', '__mod__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__rmod__', '__rmul__', '__setattr__','__sizeof__', '__str__', '__subclasshook__', '_formatter_field_name_split', '_formatter_parser', 'capitalize', 'center', 'count', 'decode', 'encode', 'endswith', 'expandtabs', 'find', 'format', 'index', 'isalnum', 'isalpha', 'isdigit', 'islower', 'isspace', 'istitle', 'isupper', 'join', 'ljust', 'lower', 'lstrip', 'partition', 'replace', 'rfind', 'rindex', 'rjust', 'rpartition', 'rsplit', 'rstrip', 'split', 'splitlines', 'startswith', 'strip', 'swapcase', 'title', 'translate', 'upper', 'zfill']

    And Ruby:

    irb(main):003:0> "some string".methods
=> [:to_java_bytes, :upcase!, :ascii_only?, :lstrip, :upto, :lines, :encoding, :prepend, :scan, :==, :clear, :squeeze!,:chop, :next!, :casecmp, :start_with?, :split, :to_f, :center, :reverse, :sub, :byteslice, :&gt;, :upcase, :next, :strip!,:count, :sub!, :hash, :bytesize, :lstrip!, :to_sym, :&lt;=, :replace, :length, :swapcase, :gsub, :intern, :succ, :capitalize, :each_codepoint, :oct, :delete!, :+, :initialize_copy, :to_java_string, :match, :unpack, :index, :rstrip!, :*, :each_char, :gsub!, :to_s, :empty?, :size, :swapcase!, :ljust, :downcase, :rpartition, :to_str, :getbyte, :sum, :crypt, :partition, :reverse!, :=~, :force_encoding, :each_byte, :tr!, :inspect, :to_c, :rstrip, :succ!, :&lt;, :[]=, :valid_encoding?, :slice!, :slice, :insert, :tr_s!, :unseeded_hash, :squeeze, :dump, :===, :end_with?, :hex, :strip, :capitalize!, :bytes, :setbyte, :chop!, :each_line, :[], :encode, :include?, :chomp!, :&lt;&lt;, :encode!, :chomp, :rindex, :to_i, :&lt;=&gt;, :eql?, :tr_s, :chars, :codepoints, :delete, :chr, :to_r, :rjust, :%, :&gt;=, :concat, :ord, :tr, :downcase!, :between?, :handle_different_imports, :include_class, :java_kind_of?, :java_signature, :methods, :define_singleton_method, :initialize_clone, :freeze, :extend, :nil?, :tainted?, :method, :is_a?, :instance_variable_defined?, :instance_variable_get, :singleton_class, :instance_variable_set, :public_method, :display, :send, :private_methods, :enum_for, :com, :to_java, :public_send, :instance_of?, :taint, :class, :java_annotation, :instance_variables, :!~, :org, :untrust, :protected_methods, :trust, :java_implements, :tap, :frozen?, :initialize_dup, :java, :respond_to?, :java_package, :untaint, :respond_to_missing?, :clone, :java_name, :to_enum, :singleton_methods, :untrusted?, :dup, :kind_of?, :javafx, :java_require, :javax, :public_methods, :instance_exec, :__send__, :instance_eval, :equal?, :object_id, :__id__, :!, :!=]