I'd Rather Not Have To Write This
Hey! Where is the blog post about moving hosting providers? Good question. Lamentably, the answer is "pilot error", A.K.A. it is all my fault. During a code push to my new hosting provider, I accidentally clobbered my new database access config file with my old config file. Fortunately, the end result of my mistake, was the loss of just one blog post.
So What Happened?
When the config was clobbered, all database requests pointed to my old hosting provider and when my service time with them expired, they removed my former database from their systems.
As a result, any information that was added to the database after the clobber was lost (with the exception of the RSS PHP post which I still had on my computer in markdown format). sql
So What Was Lost?
The decision making criteria that I used when selecting http://www.webfaction.com as my new hosting provider and http://gandi.net as my new domain registrar is gone. I'll do a quick recap:
webfaction
- multiple websites on one hosting account
- shell access
- runs web apps and not just scripts
How about I make this nice and short and just send you to their features page
Gandi.net
- They are not GoDaddy
- No Bullshit is there motto
- They support project that I like
And who could forget the loss of information about http://googlebutter.com? Not me, that's for sure. Hi Buddy!
What Am I Doing to Limit Loss in the Future?
The real culprit here is me for not making enough database backups. So I had better write a script to make database backups for me. For security reasons, my new provider does not allow mysql connection from other than localhost or 127.0.0.1, I need to:
- Create an SSH tunnel to my new host
- Run a mysqldump and pipe the output to a text file
- tar gzip the text file
Enter the Ruby
require 'date'
require 'open3'
#create variable for database access
db_server = "MYSERVER_IP_ADDRESS"
db_user = "MY_DATABASE_USER"
db_pass = "MY_DATABASE_USER_PASSWORD"
#what day is it?
Ymd = Date.today.strftime("%Y%m%d")
sql_file = "#{Ymd}_dump.sql"
#create an ssh tunnel
sshcmd = "ssh -L 3307:127.0.0.1:3306 #{db_server} -N"
dumpcmd = "mysqldump -A -u#{db_user} -P3307 -h127.0.0.1 -p#{db_pass} > #{sql_file}"
#start the sshcmd
puts "starting ssh tunnel..."
Open3.pipeline_start(sshcmd) do |threads|
#sleep for a bit and give the thread time to connect
sleep 10
#get the first thread
t = threads[0]
puts "ssh tunnel has pid: #{t.pid}"
#run the dump command
puts "dumping database info"
system(dumpcmd)
puts "data dump is complete, killing ssh tunnel pid #{t.pid}"
#kill the ssh tunnel
Process.kill("TERM", t.pid)
end
#tar the sqlfile
tarcmd = "tar -czvf #{sql_file}.tgz #{sql_file}"
system(tarcmd)
system("rm #{sql_file}")
For easy copy paste, the code is in the hoof http://hoof.jezra.net/snip/nT
What I should really do, instead of tunneling the mysqldump and then tar gzipping the data on my machine, Is perform the dump and archive on the server and then transfer the compressed archive to my machine.
Now quit reading, and go backup your data! (Then go see my buddy at http://googlebutter.com) Hi Buddy! hahaha
Quite often I find myself over at http://thesession.org checking out old traditional tunes. The site offers the tunes as sheet music and ABC Notation. While I certainly appreciate having ready access to the tunes, I find that, more often than not, I need to copy the ABC notation into a text file, convert the text file into a [lilypond[(http://lilypond.org) file, transpose the lilypond file to the G Major scale, and then finally add some code that will display the tune as banjo tabs. Oh man, did you catch all of that? There has to be an easier way to do this.....
Oh, there is...
For the most part, the process is just a series of text manipulations which can be accomplished in just about any programming language. Since I am currently on a Ruby kick, I figured I would just hammer out a quick script to automate the process of retrieving ABC notation from a given page and converting the results into something more usable.
Enter the Ruby
require 'optparse'
require 'open-uri'
require 'rexml/document'
require 'iconv'
def sanitize( string )
#replace non alpha-numerics with _
string = string.gsub(/[^A-z0-9_]/,'_')
return string.downcase
end
def error( string )
puts "**** ERROR ****"
puts string
exit
end
option = {}
OptionParser.new do |opts|
opts.banner = "Usage: sessionconvert [options] [thesession.org_URL]"
opts.on("-b", "--banjo", "add banjo tabulature") do |v|
option[:banjo] = v
end
opts.on("-l", "--lilypond", "compile with lilypond") do |v|
option[:lilypond] = v
end
end.parse!
url = ARGV[0]
# was a url supplied?
unless url
error "you must supply a thesession.org url"
end
begin
text = open(url).read
rescue
error "Failed to read #{url}"
end
begin
#watch out for sneaky
ic = Iconv.new('UTF-8//IGNORE', 'UTF-8')
valid_string = ic.iconv(text)
doc = REXML::Document.new valid_string
rescue
error "Failed to parse #{url}"
end
root = doc.root
p = root.elements["body/div[@id='abc']/div[@class='box']/p"].to_s
abc = p.gsub(/<.*>/,'')
#get the title
m = /T: (?<title>.*)/.match( abc )
title = m['title']
#get the key
m = /K: (?<key>.*)/.match( abc )
key = m['key']
#sanitize the title, we will use this a bunch
sanitized_title = sanitize(title)
#what will the filenames be?
filename_abc = sanitized_title+".abc"
f = open(filename_abc,'w')
f.write(abc)
f.close
begin
`abc2ly #{filename_abc}`
rescue
error "abc2ly is not installed"
end
#what will the lilypond file be?
filename_ly = sanitized_title+".ly"
if option[:banjo]
#what is the key we are transposing from
t_key = key[0].downcase
banjer_bit = "\\transpose #{t_key} g,
\\context TabStaff = \"banjer\"{
\\set TabStaff.stringTunings = #banjo-open-g-tuning
\\set Staff.instrumentName = #\"Banjo\"
\\context TabVoice = \"banjer\" { \\tabFullNotation \\stemDown \\voicedefault }
}"
text = File::read(filename_ly)
#substitute the banjer stuff
text = text.gsub(/<<.*>>/m, "<<\n #{banjer_bit}\n\t>>")
#make a new lilypond file
filename_ly = sanitized_title+"-banjo.ly"
#write the new text to file
f = open(filename_ly,'w')
f.write(text)
f.close
end
#did the user request compiling
begin
`lilypond #{filename_ly}` if option[:lilypond]
rescue
error "lilypond is not installed"
end
If you have problems copying the code, please see http://hoof.jezra.net/snip/nQ
Usage
Since I don't always want to convert a tune to banjo tabs (some tunes are better on the concertina), a flag (-b) needs to be included with the arguments to specify that the tune should be converted to banjo tabs. Similarly, including the -l flag with compile the final lilypond file into a pdf of sheet music or tabs.
For example: The tune Rakes of Mallow is found at http://www.thesession.org/tunes/display/85 and to convert the tune to banjo tabs and compile to a PDF the following command would be used.
This is what the final PDF looks like. Damn that's nice!
...and this is me butchering the tune on my banjo. Damn, I need more practice.
Awesome! Now quit reading, and go butcher a tune on the instrument of your choice.
A few years ago, I decided to rip and organize my entire CD collection. The resulting music files are all on player a small computer, running MPD, and connected to my home stereo. It took me a while, but it was worth it. In a similar fashion (since I am somewhat stuck at home recuperating) I figured now would be a prime time to rip my entire DVD collection. To accomplish this feat, I am using a piece of software called handbrake.
More specifically, I am using the command-line version HandBrakeCLI with a small bit of Ruby code.
The Code
require 'open3'
input = '/dev/sr0'
title = ARGV[0]
#what are the video options?
v_opt = "-r 29.97 -b 1200 -2 -T"
#what are the audio options?
a_opt = "-E lame -B 128 -R 48 --mixdown stereo"
#what are the source options?
source = "--main-feature"
command = "HandBrakeCLI -i #{input} #{source} #{v_opt} #{a_opt} -o #{title}.mp4"
puts command
Open3.pipeline( command )
Why the Ruby code?
the Ruby code takes one argument: the name of the movie, and creates a string representing the command I want to use for ripping the DVD.
So then one might ask why not just use the graphical version of handbrake? Good question, I'm glad you asked.
I don't just run the Ruby script ( by the way, I named the script 'hbwrap.rb' ). The script is run with a few other scripts as well. Remember the text to speech thingy from a while ago? Well I run that as part of the command as well.
The process is as follows:
- Put Carts of Darkness into the DVD drive of my bedroom computer.
- On the command line, run: hbwrap.rb Carts_of_Darkness && ttsender "finished ripping" && eject
- Walk into living room and do some work
- Wait until the stereo says "finished ripping"
- Walk into the bedroom and repeat the process
- ?
- Contemplate purchasing a much much larger hard-drive to hold all of my movies
- Think of a way to playback the movies that is controllable by an NES controller
- Write a blog post about the process
Yes, I have already ripped Jaws the Revenge. By accident, I almost ripped The Princess Bride, but I decided to wait until I have time to edit out all of the useless Fred Savage crap that ruined the film. In case you were wondering, The Princess Bride is one of the few instances where the book is worse than the film.
Now quit reading, and go do something something something.
(isn't it legal now to bypass this to backup your own collections and whatnot?)
I *think* what I was saying is that I thought handbrake was just a transcoder, not a ripper+transcoder.
it's awesome that it is both!
While hacking code and introducing bugs to a project, I sometimes find it necessary to search for a string in every file in a directory. For example, suppose I'm pulling my hair out trying to find where in a drupal project a certain function is called. It could be in a module, or a theme, or the template file. Egads! the code is all over the place and I most certainly do not want to open every damn file and search for the function.
I had been using a script called 'find_in_dir' to help me find a string in a directory and the script is as follows:
The usage of the script was simple:
[jezra@fatlappy ~]$ find_in_dir PATH_TO_SEARCH STRING_TO_FIND
The script would recursively search each file in the directory and output the lines in a file containing the search string and then output the path to the file that was searched. While this definitely did what I needed, I found that I was spending too much time tracking down the path to the file because all of the output was just plain text on the screen.
What I really wanted/needed was something to differentiate the file path in the output from the rest of the output, so I fired up geany and hacked together some Ruby code to do what I need (and output the file path using green text. ohhhhh fancy!)
Enter the Ruby
#make a helpy helper function
def help()
print "--USAGE--\n"
print "findindir directory string_to_find\n"
end
#find (
#loc: path to a file or directory,
#string: the string to search for
#)
def find(loc, string)
#is the location a directory?
if File.directory?(loc)
#loop through each item in the directory
Dir.foreach(loc) do |name|
#ignore . and ..
if name!='.' and name!='..'
#what is the path of the item?
path = File.join(loc,name)
#recurse
find(path, string)
end
end
else #this is a file
#by default, we state that there is no match to the string
match = false
#loop through each line in the file
File.foreach(loc) do |line|
#does the line contain our string?
if line.include?(string)
match = true
puts line
end
end
#a match was found
if match
puts "\e[32m#{loc}\e[0m\n\n"
end
end
end
#set some vars based on user input
start_loc = ARGV[0]
string = ARGV[1]
#did the user enter enough data?
if start_loc.nil? or string.nil?
help
else
#determine the absolute path to the start location
path = File.absolute_path(start_loc)
#find that shit!
find(path, string)
end
Having trouble copying the code? get it at http://hoof.jezra.net/snip/6
The code isn't perfect and could certainly use some improvements, such as:
- ignore broken symlinks
- output the line number where matches were found
- default to current working directory when the user doesn't specify a directory
It may not be the best, but it's a start. Now quit reading and go find something.
find $1 -type f -exec grep -i $2 {} \; -printf "[%p]\n" | colorit
It has the side effect that it might also colorize other parts of the results, and I didn't quickly see a way to safely remove the brackets after colorization.
grep -ir --color=always pattern directory
the -r recurses through all subdirectories, and --color=always colorises the output.
The Need
For part of a Ruby project I've been working on, I needed a way to play audio files, and since I am using the project to teach myself the basics of Ruby programming, I decided to write my own audio player class utilizing the Gstreamer Multimedia framework.
Having written what is essentially the same code in both Vala and Python programming languages, I thought this would be fairly simple with the aid of the documentation.
RTFM (a rant)
The documentation for using Gstreamer with Ruby is part of the Ruby Gnome2 documentation and I found it to be dreadful to use. Since it is difficult to actually find a link to the Gstreamer related documents, I'll include a link http://ruby-gnome2.sourceforge.jp/hiki.cgi?Ruby/GStreamer
Honestly, I tried to read the documentation and it was so frustrating that I started to hate Ruby. What bothered me the most about the documentation wasn't the abundant amount of missing information, it was the 500 Server Error that I would see every 4 out of 5 clicks. Why someone thought it would be a good idea to server the files as a CGI wiki and not as good old static files is beyond me. Aaahhhhhhh! I hate that crap!
OK, time to relax and just look at some code.
Enter The Ruby
require 'thread'
require 'gst' #gem install gstreamer
#the gst namespace is Gst
#initialize gst
Gst.init
class Player
def initialize()
#create a thread for a glib main loop
thread = Thread.new() do
@mainloop = GLib::MainLoop.new
@mainloop.run
end
#make a few queries
@query_position = Gst::QueryPosition.new(Gst::Format::TIME)
@query_duration = Gst::QueryDuration.new(Gst::Format::TIME)
#make the playbin
@playbin = Gst::ElementFactory.make("playbin")
#get the playbins bus
bus = @playbin.bus
#watch the bus for messages
bus.add_watch do |bus, message|
handle_bus_message( message )
end
end
#we will need to get the current position and duration of the playbin
def position_duration()
begin
#run the query
@playbin.query( @query_position )
#what is that, picoseconds? I'll take milliseconds thank you.
position = position = @query_position.parse[1] / 1000000
@playbin.query( @query_duration )
duration = @query_duration.parse[1] / 1000000
rescue
position = 0
duration = 0
end
return {'position'=>position,'duration'=>duration}
end
def status()
#get the state
bin_state = @playbin.get_state
#isn't there a better way to convert the state to a string?
case bin_state[1]
when Gst::State::NULL
state = 'NULL'
when Gst::State::PAUSED
state = 'PAUSED'
when Gst::State::PLAYING
state = 'PLAYING'
when Gst::State::READY
state = 'READY'
end
volume = @playbin.get_property("volume")
uri = @playbin.get_property("uri")
pd = position_duration()
#return state, volume
status_hash = {'state'=>state, 'volume'=>volume, 'uri'=>uri}
#add the position and duration to the hash, and return
return status_hash.merge( pd )
end
#set or get the volume
def volume(val)
if !val.nil? and val>=0 and val<=1
@playbin.set_property("volume", val)
end
return @playbin.get_property("volume")
end
def seek_percent(val)
if !val.nil? and val>=0 and val<=1
pd = position_duration()
duration = pd['duration']
if duration > 0
seek_loc = val*duration * 1000000
seek = Gst::EventSeek.new(1.0, Gst::Format::Type::TIME, Gst::Seek::FLAG_FLUSH.to_i | Gst::Seek::FLAG_KEY_UNIT.to_i, Gst::Seek::TYPE_SET, seek_loc , Gst::Seek::TYPE_NONE, -1)
@playbin.send_event(seek)
end
end
return position_duration()
end
def quit()
@playbin.stop
@mainloop.quit
#I thought no one liked a quitter?
end
def set_uri(uri)
#null the playbin state
@playbin.set_state(Gst::State::NULL)
#set the uri
@playbin.set_property("uri",uri)
end
def play()
#really? just play
@playbin.play
end
def pause()
#really? just play
@playbin.pause
end
def handle_bus_message( message )
case message.type
when Gst::Message::Type::ERROR
#null the pipeline
@playbin.set_state(Gst::State::NULL)
#TODO: send a signal that playing is finished
when Gst::Message::Type::EOS
#null the pipeline
@playbin.set_state(Gst::State::NULL);
#TODO: send a signal that playing is finished
when Gst::Message::Type::TAG
tag_list = message.parse()
#we need to get the key and value from the tag
tag_list.each do |key,val|
#TODO: store some of this data
end
when Gst::Message::Type::STATE_CHANGED
state = @playbin.get_state
else
#what should we do?
end
#return true or shit breaks: why is this?
true
end
end
if \_\_FILE__ == $0
input = ARGV[0]
if input.match(/^http:\/\//)
#why the hell doesn't this work?
uri = input
else
uri = "file://"+File.absolute_path(ARGV[0])
end
player = Player.new
player.set_uri(uri)
player.play()
loop = true
sleep 1
while loop
puts "type 'quit' to quit"
s = $stdin.gets.chomp
if s.eql? "quit"
loop = false
end
end
player.quit()
end
For some reason the code will not play an audio file over HTTP and this bothered me for a bit, then I decided that I just don't care. One thing you may notice is that this class will create a new thread for running a GLib mainloop. Had this class been part of a larger project that uses a GLib mainloop, the new thread probably wouldn't be necessary, but hey, I'm not writing a GLib based project.
You may want to use "playbin2" instead of "playbin".
The Need
Today I started putting together a little image gallery for my sideburns and I really wanted a way to easily scale a directory of images to a specific height, save the result in a directory and then repeat the process and save in yet another directory.
The Solution
There is probably an easy way to do what I want using GIMP, but I thought it would be better if I wrote a script to handle this for me. Technically, the script that I wrote is just a fancy wrapper for the imagemagick convert command.
Enter the Ruby
#create a function to end the script with a message
def exit_help(message=nil)
#did a message get passed in?
if !message.nil?
#print the message
puts message
end
#print the 'help'
puts "--Help--
#{$0} path/to/image/directory thumbnail_height image_height
"
exit()
end
#make a function to create a dir if it doesn't exist
def check_dir(dir)
if !File.directory?( dir )
Dir.mkdir(dir)
end
end
#make a function to run a set command
def do_command(comm)
puts comm
system(comm)
end
#set some variable based on the input from the user
start_dir = ARGV[0]
thumb_height = ARGV[1].to_i
image_height = ARGV[2].to_i
#if any of the vars aren't set, the user didn't input enough info
if start_dir.nil? or thumb_height==0 or image_height==0
exit_help()
end
#does the start dir exist?
if !File.directory?( start_dir )
exit_help( "#{start_dir} does not exist" )
end
# define the output dirs
gallery_dir = File.join(start_dir,'gallery')
thumb_dir = File.join(start_dir,'gallery_thumbs')
# make the dirs if they don't exist
check_dir(gallery_dir)
check_dir(thumb_dir)
#loop through the files in the start_dir
start_dir_files = File.join(start_dir,"*.{png,jpg}")
Dir.glob(start_dir_files) do |file|
#what is the name of the file?
fname = File.basename(file)
#what is the path of the new gallery file?
gfile = File.join(gallery_dir,fname)
#what is the path of the new thumbnail file?
tfile = File.join(thumb_dir,fname)
#what imagemagick commands will resize the images?
c1 = "convert -resize x#{image_height} #{file} #{gfile}"
c2 = "convert -resize x#{thumb_height} #{file} #{tfile}"
#run the commands
do_command( c1 )
do_command( c2 )
end
What It Do?
I'll tell you what it do! When run, the script requires three arguments: 1. The name of the directory containing the images 2. the height of the thumbnail image 3. the height of the gallery image
The end result will be a directory named "gallery" and a directory named "gallery_thumbs" in the images directory. Gallery will contain the images resized to the gallery height specifications. I'll leave it to you to figure out what images are in the "gallery_thumbs" directory.
My Take Away
Originally, I had planned on using the exec() function to run my convert commands but exec() replaces the calling script with the command being called. This means that as soon as I ran exec(), my code would quit running. Oh well, system() works just fine for my needs.
The Need
When I'm away from home, I often find it rather useful to connect to my home network. Due to the dynamic IP address assigned by my Internet Service Provider, I use a third-party service to map a domain name to my dynamic address. This scenario requires that a machine in my network notifies the third-party service of network address changes; and a script to do this was fairly easy to write.
The Preparation
The first thing I need to do was create an account with a dynamic IP Domain Name Service provider. Previously I had use http://dyndns.com but their service has been intermittent for the last few days so I though a switch to http://no-ip.com was in order.
Aside from providing Dynamic DNS, No-IP provides software allowing Linux, Macintosh, and Windows computer users to easily update their dynamic address information. Even more importantly, No-IP provides an API for programmers to utilize when writing their own software to update address information.
The Script
Honestly, this could be done with a single line curl command, but that wouldn't be fun and it wouldn't help me learn Ruby. Enter The Ruby
require 'open-uri'
#set up some variables
u = "MYUSERNAME"
p = "MyPaSsWoRd"
h = "my.hostname.com"
url = "https://dynupdate.no-ip.com/nic/update?hostname=#{h}"
#open the url
f = open(url,
"User-Agent" => "Jezra's No-IP Update Script",
:http_basic_authentication => [u,p]
)
#output the return data
puts f.read()
This script is now happily running on my miniserver at 4 hour intervals.
The big take away here for me, was learning how to use open() from the open-uri module to set the User-Agent of a request as well as utilizing http basic authentication.
One More Thing
Since I'm learning Ruby, and since I learn by doing, I force myself to write (semi)useful utilities for everyday task, like creating a password for my No-IP account. Instead of just coming up with something I'll remember, like 1 2 3 4 5, I decided to write a string generator. woohoo!
def help()
puts "--Hey, you are doing it wrong--
#{$0} requires a numeric argument
examples:
# create a 20 character string
#{$0} 20
# create a 10 character string
#{$0} 10
"
end
#what chars do we have to pick from?
chars = "!@$%^*_+=ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"
#get the first argument as an integer
num = ARGV[0].to_i
#the num better exist, and it better not be zero!
if num.nil? or num.zero?
help()
exit()
end
charslen = chars.length
string = ''
(1..num).each do
char = chars[ rand(charslen) ]
string << char
end
puts string
Sweet! Now quit reading, and go learn something new.
Hello Ruby
Recently, I've been looking into the Ruby Programming Language and like most programmers I started by writing a "Hello World" program. Now let me say that I am not a fan of most Hello_World programs simply because I don't think they really give much information to the viewer of the code, so I thought I should share the code that I wrote.
There were a few programming language features that I thought should be included in my code.
An Object Oriented class
Quite a bit of my coding is Object Oriented and since most anything I program in the future using Ruby will be Object Oriented, this was a must.
A class that extends the functionality of the previous class
Writing my own classes is good and dandy, but very often I need to extend a base class from and included code library with additional functionality and I wanted to see what sort of coding requirements Ruby has for extending classes.
some sort of array thingy
Each programming language has various names for ways to store a collection of data: list, dictionary, hash table, array, and a few other things that I usually forget because I don't pay much attention to nomenclature. Anyway, I always end up having to program some way to deal with a list of words or numbers or something, so I made sure to include an array in my program.
Enter the Ruby
#define a class to print out some words
class PrintThing
def initialize
#we need a variable to hold the words to be printed
@words = [] #use an empty array
end
#define a function to set the @words
def set_words(words)
@words = [words]
end
#define a function to output all of the elements of the words array
def print_words()
puts @words.join( " " )
end
end
#extend the printthing class to be more advanced
class AdvancedPrintThing < PrintThing
#define a function to add an element to the words array
def add_word( word )
@words.push( word )
end
#define a function to truncate the words array
def clear()
@words = []
end
end
if __FILE__==$0
#create an instance of the PrintThing
pt = PrintThing.new
#set some words
pt.set_words( "Hello World" )
#print the words
pt.print_words()
#create an instance of the AdvancedPrintThing
apt = AdvancedPrintThing.new
#set some words
apt.set_words( 'Howdy Buddy!' )
#print the words
apt.print_words()
#clear the word list
apt.clear()
#add a bunch of single words
apt.add_word( 'How' )
apt.add_word( 'are' )
apt.add_word( 'you?' )
#print the words
apt.print_words()
end
What I learned
- Defining a class is easy and a new class should have a constructor method named "initialize" if the class needs to do something when a new instance of the class is created
- Extending a class can be done with the "<" symbol, so to code that class A extends class B is as simple as "class A < B".
- a variables scope is determined by the variable's starting character.
$ for global variables
@ for instance variable
@@ for class variables
While this can look confusing at first, I like being able to determine at a glance exactly what the scope of a variable is. - There was no need to worry about code indentation. Ahhh relief.
- Ruby does not create byte-code when an ruby program is run. sigh of disapproval
All things considered, I like the language and there is some rather impressive documentation available at http://www.ruby-doc.org/
Now quit reading, and go find yourself a programming project.
http://mislav.uniqpath.com/poignant-guide/book/chapter-1.html