Archive for the 'linux' Category

Backup your Backups (Subversion)

Its interesting that while source code repository like Concurrent Versions System (CVS) and Subversion (SVN) are primary used as for source code sharing and revision management, one useful purpose for these repositories are actually useful for backups. Its like using using Google Docs not only for online collaboration, but to keep your documents in the cloud in case you lose them.

An interesting twist at work is that I have to backup the subversion database in case Subversion fails- here we are backing up the backups.

The power and freedom in linux usually means that often no one does a single job a single way. So, here’s my take on writing a script used for saving my svn databases.

# Backup SVN Script
DIR=/svn # Svn directory
DATE=$(date +%Y%m%d) # Date
BACKUPDIR = /backups

cd $BACKUPDIR
# Clear a temporary directory
rm -rf dumps/*
# Creating a list of SVN directories
for i in $(ls -l $DIR | awk '/^d/ { print $NF }') ; do
echo $i
# Export each SVN database
svnadmin dump $DIR/$i > dumps/$i.dump
done
cp $DIR/* dumps/ 2> /dev/null
# Taring them
tar zvcf archive/svn$DATE.tgz dumps/*
# Copying to another safe location
cp archive/svn$DATE.tgz /another/secret/location

Brief explaination of the script, if the script and comments doesn’t speak enough. All svn databases are dumpped in svn export/import friendly formats which are then zlib tarred.

Lets assume one day Murphy’s Law take places, your entired svn database vanish into thin air but your backups in the safe secret location is intact. Here’s how you could restore the subversion database easily, assuming your SVN servers are up and running.

# load_repo_after_the_sky_crashes.sh
tar zvxf $1.tgz # extract the backups
for i in $(ls -l dumps | awk ‘/^d/ { print $NF }’) ; do
echo $i
svnadmin create $i
svnadmin load $i < $1.dump
# chown / chmod file system permissions if needed,
done

The method for backups and restore is similar to migrating SVN repository and here’s a post if you wish to read up on them.

The backup script may be called by a cron job and run periodically like daily. To prevent taking too much disk space, you may decide to keep a number of the latest backup copies and delete older backup copies. Here’s a line in bash you could do for this.

ls -t * | tail -n +10 | xargs rm -rf # Delete everything except the 9 latest files

Who backup the backups like who guard the guards?

Compiling Python Scripts

Not too sure how much this would help in debugging python scripts, but at least it tells you any syntax errors before a long script runs for an hour before throwing an error.

Here’s the code

# compile.py
import py_compile, sys
py_compile.compile(sys.argv[1])

Here’s a sample python script

# helloworld.py
print “Hello World!”

Here’s an example to compile it
$ python ./compile.py ./helloworld.py

Running this generates a bytecode file called helloworld.pyc which could be run
$ python ./helloworld.pyc

python could also be used with the flag “-O” for optimize generated bytecode

>>> exit()

Shell Script for Looping Each Directory

Let’s say you are creating a batch script to loop through each “folder” you have in an directory- run:

$./list_dirs.sh /app

inside the shell script:
#list_dirs.sh
for i in $(ls -l $1 | awk ‘/^d/ { print $NF }’) ; do
echo $i # replace with actions to be done
done
#end script

basically ls -l will display all files+folders with their attributes.
awk filters to those lines starting with d attribute (/!d/) and strips out to the last field ($NF)

blog>logout

My Networking Bridge is Back on FreeBSD

I’m glad.

Fixing Bridge
Fixing the bridge in the wee hours- (laptop on the left, pc to be on the bridge in the middle, and console of my BSD server)

I managed to pinpoint the fault and get my network bridge working on the pentium II server again, and the isolated pc in my room is connected to the internet again.

So the PCI slot is damaged, after multiple reboots, and network configuration. So the network card got swapped to another PCI slot.
Pii Server

Here’s what I used to bridge the connections.


ifconfig bridge create
ifconfig bridge0 addm fxp0 addm rl0 up
ifconfig fxp0 up
ifconfig fl0 up
ifconfig fxp0 0.0.0.0 255.255.255.0
ifconfig rl0 up 0.0.0.0 255.255.255.0
dhclient bridge0

So with this I avoided the migration of the server to the pentium 3 hardware, and delay installation of CentOs. Of course, there are tricky issues in FreeBSD, like a changing MAC address for the bridge each time its created. But this all I would like to say for now. If I may, I would go into the details of FreeBSD and the startup configuration scripts like /etc/rc.conf next time.

Java Ftp Client

So for the 2nd part of my networking assignment, we have to create a simple FTP client.

Screenshot of my Worked-In-Progress FTP Client
zz85 Ftp Client

I was hoping…
to create a web server vs a ftp client as I thought
1. a web server could be more practical, simpler to implement, but could pack more features
2. the ftp protocol is old and troublesome, I already spent quite some time in the past trying to understand the protocol, lots of ftp clients exist already

Nevertheless, given the low weightage of the assignment, I didn’t want spend much time on it.
Lots of other work need more focus, but my programming habbits and “chiong project” culture in polytechnic kicked in and I did a little more than the requirements.

Although I believe I must have did quite some amount of work on Swing and networking component, bad memory and the rust affected me and I found myself using some time looking up various stuff. Sometimes I think, “are we letting Google is do all our work?”

Inspired by…
Anyway the product of my implementation was inspired by many many other stuff.
Filezilla as a free good implementation of FTP client. Command line ftp, telnet software. The terminal. Therefore the black screen.
Port sniffers- therefore the debug windows. IDE/Frontends thats why the GUI on the left.

I created a client in Eclipse then migirated to Netbeans trying their new 6.0 Beta1 (and Java 6- I’m must have lagged 2 versions since poly)

My product features…
Created for academic reasons, this software has educational purpose. Raw commands allowed to be typed in quickly on the right side of the screen. Some commands emulate the command line ftp software.

The way is screen is split is also to represent the User-FTP, PI, and DTP architecture as described by the RFCs. (RFCs 959 and rfc1579 were read). It does some upload and download.

Alright the client doesn’t does much, its far from perfect or good, but it shows my attempt trying to be creative.

Download it
here

Try it…
Just double click. If you wish to run from a command line, use
java -jar FtpClient.jar or java -cp FtpClient.jar ftpclient.FtpClientUI

Happy Messing with FTP.