31 December, 2008

Video Editing Ninja Moves with Linux

Our program focus for the past few months has concentrated on advanced image processing algorithm development. Pre-collected video streams are utilized as inputs to our development process, leading us to become familiar with Linux video conversion utilities. I'll briefly identify some of the features we've encountered that were useful to us.

Source Stream
You need to first start with a collected video stream. There are numerous methods of gettin' hold of an input stream: download it, extract it from a dvd, collect if from your favorite Tivo device, etc.

To extract an uncompressed video stream into an Avi container from a dvd title:
Note, we are attempting to preserve the highest level of video quality by preventing lossy compression.

$ ffmpeg -i /media/cdrom/VTS_01_1.VOB -vcodec rawvideo /tmp/source.avi


An alternative (Tivo'lious) video stream can be extracted from a Tivo Mpeg-Ps container. A SourceForge project Tivo File Decoder provides the means of extracting the Mpeg video stream from the proprietary Tivo file container.


$ tivodecode --mak 0012345678 -o /tmp/source.mpg ~/Desktop/BigBangTheory.TiVo


We'll use the Duke as our source video stream:


We can get a few details of the video file by using the file *nix command:

$ file /tmp/source.avi
/tmp/source.avi: RIFF (little-endian) data, AVI, 720 x 480, ~30 fps, video:, audio: (stereo, 48000 Hz)

Of particular interest are the frame dimensions 720x480 and 30fps frame rate. These details will be useful later in examining the effects of some of our commands.

Frame Extraction/Reassembly
Extracting frames from a source video stream is useful for many possible reasons. Extracted frames can be stored in a series of Ppm or Jpeg image files.












Extracting frames to Jpg image files can be done by:

$ ffmpeg -i /tmp/source.avi %08d.jpg


Extracting frames to Ppm image files done by:

$ ffmpeg -i /tmp/source.avi %08d.ppm


And extracting frames to Pgm image files can be done by:

$ ffmpeg -i /tmp/source.avi %08d.pgm

You can reassemble the video element of the source video by assembling the extracted images back into a video stream. In order to preserve the framerate of the source video stream you need to know and specify the frame rate of the encoded video. If you don't specify the original frame rate, a default of 25fps will be used which may result in a faster or slower playback than the original.

$ ffmpeg -r 30/1 -i %08d.jpg videoOut.mpg
The resultant video is a near visual duplicate of the original, the sound channel however was eliminated as a result of the disassembly/reassembly procedure.

Frame Rate
Re-sampling the source video into an alternative frame rate is advantageous.

The original source video was sampled at 30fps, to sample the destination at 5fps you'd command:

$ ffmpeg -i /tmp/source.avi -r 5/1 /tmp/videoOut.avi


To upsample to 60fps you'd command:

$ ffmpeg -i /tmp/source.avi -r 60/1 /tmp/videoOut.avi

Frame Cropping
Faking a letterbox can be emulated by cropping the top and bottom off the original video. Cropping the top and bottom 160 rows can be done by:

$ ffmpeg -i /tmp/source.avi -croptop 160 -cropbottom 160 /tmp/letter.avi
$ file /tmp/letter.avi
/tmp/letter.avi: RIFF (little-endian) data, AVI, 720 x 160, ~30 fps, video:, audio: (stereo, 48000 Hz)


Video Segments
Snaggin' a clip out of a source video is always useful. To extract the Dukes classic line you can specify the starting position and duration of the clip. Specifying a starting location using the -ss specifier followed by the -t duration specifier can be done as follows:

$ ffmpeg -ss 85 -t 18 -i /tmp/source.avi /tmp/clip.avi



Altering Video Dimensions
It is sometimes advantageous to change the video dimensions of the source video. If high resolution imagery isn't required changing the video stream to a smaller dimension reduces the file size and expedites processing.


$ ffmpeg -i /tmp/source.avi -s 320x240 /tmp/small.avi
$ file /tmp/small.avi
/tmp/small.avi: RIFF (little-endian) data, AVI, 320 x 240, ~30 fps, video:, audio: (stereo, 48000 Hz)

Video Format Conversions
Converting to various file formats and codecs is the real intention of ffmpeg. Ffmpeg allows conversions to pretty much any popular (and many unpopular) containers and encoders. Below are a few of the popular formats we used, many others are available.


$ ffmpeg -i /tmp/source.avi /tmp/source.mpg
$ file /tmp/source.mpg
/tmp/source.mpg: MPEG sequence, v1, system multiplex

$ ffmpeg -i /tmp/source.avi -vcodec rawvideo /tmp/source.avi


The raw video ensures the highest quality of video, useful if you wish to preserve the highest degree of resolution of the original video. An alternative is to specify -qmin 1 and -qmax 1 which specifies a range 1-51 (1-highest quality).

30 November, 2008

Flash Player Installation on Debian 4.0 AMD64 Distro

I recently purchased a new computer, downgrading my now 4 year old processor to the closet. I was pleased to install my first 64-bit operating system, but found it a challenge installing a compliant flash-player plug-in. A breezy Google search resulted in the following Clif-notes:



# wget -O - http://backports.org/debian/archive.key | apt-key add -



# vi /etc/apt/sources.list

+ add the following line:
deb http://www.backports.org/debian etch-backports main contrib





# apt-get update
# apt-get install flashplugin-nonfree

16 September, 2008

Tcl Package Example

I often struggle setting up a Tcl script that utilizes packages. For convenience I'm publishing a template for scripts that capitalize on packages.


::::::::::::::
process
::::::::::::::
#!/usr/bin/tclsh


proc log { msg } {
set lvl [expr [info level] -2]
set indent [string repeat " " $lvl]
tclLog "$indent\___$msg"
}

proc traceLog { } {
set uplevel [expr [info level] -1]
set callSig [info level $uplevel]
set el [split $callSig]
log "call trace: [namespace current][lindex $el 0]([lrange $el 1 end])"

}

proc packageInit { } {
upvar #0 auto_path auto_path

log "initializing system"
set pathName [file dirname [info script]]
log "discovered path name '$pathName'"
pkg_mkIndex $pathName tcl/*.tcl
lappend auto_path $pathName
log "done initializing system"
}

proc proc01 { a b c d } {
traceLog
}

proc proc02 { } {
traceLog
proc01 2 4 6 8
}

#-----main-----
packageInit
package require package01
package01::function01

log "main thread initializing"
proc01 1 2 3 4
proc02
log "main thread terminating"


::::::::::::::
tcl/package01.tcl
::::::::::::::
package provide package01 1.0

namespace eval package01 {
proc function01 { } {
puts "...function01() method entry"
puts "...function01() method exit"
}
}

28 June, 2008

Comical Techno Cast

I regularly watch CommandN which weekly identifies webpicks of the week. Last week they suggested TheCircuit; I just watched my first episode and it was hilarious. The host is priceless.

24 June, 2008

Oompa Loompa Of Science

Speaking from an oompa loompa, quite possibly the funniest quip broadcast on the Big Bang Theory.



Sure, it's a cop-out posting a video in a blog...but I haven't posted in some time, and this is funny stuff.

03 June, 2008

A Flawless Demo

Exactly 5 hours and 11 minutes ago our team facilitated a flawless demo. In my opinion, it couldn't have went better.

This is significant for us because, as I may have mentioned in the past, our program funding is dwindling out, soon to expire by years end. So, we got all the right people in the room; our IPT lead, the director of our division, the business development leads for the division, the IRAD lead for the division....all the big ticket players who we could coax into a meeting.

The entire audience was very excited about the technologies, the customer satisfaction, the performance.....everything. All was going well, 'til the discussion came to funding.

Then, all these major players dropped their eyes to the floor and acted like they "left their wallet at home". Sigh.

13 May, 2008

The Secret To My Success

I've now been in the software engineering industry for 10 years (as of the upcoming 18th of May). Through those 10 years, I've received three progressions; read that as promotions.

As much as I'd like to take full credit for these achievements, achieving each with my charm, brilliant mind and overachieving attitude; if I'm honest, one of the major factors has been my luck in my assignments to managers.

Right out of the gate, fresh out of school I was assigned a military-grade, Lt Colonel in the Minnesota National Guard. Within the first couple months, he was asking career-oriented questions that I certainly wasn't ready to answer; questions like "where are you going to be in five years?", "do you want to lead a team?", "did you watch the Twins game last night?". Ok, the last one doesn't apply, but you get the idea. Through the years I've had my share of managers, each with strengths and short-comings, but all-in-all I've got a pretty good roll of the dice in the management department.

It's important to think about career-oriented questions early in your career, and you should revisit them continuing throughout your career. If you don't know where you want to go, there isn't much a chance you'll ever get there.

Aside from inspiring a career direction, leads are critical to achieving your career goals. Fact of the matter is, you need someone in your corner to put you up for promotions, to champion and defend you against the progression committee. Most organizations offer a fixed number of promotions and grades each year, and you need someone to go to bat for you in telling the committee why you deserve it more than the next guy. It's a competitive world, and these are the rules of the game.

A good number of my colleagues have the same skills as I do, the direction and drive similar to mine, but have come up short in manager assignment. Some of them are 2-3 promotions behind me as a direct result. Some, a consequence of unstable program assignments which resulted as mid-year changes in management, a stake-to-the-heart of any possible promotion that given year.

If you take anything from this entry, consider these two key points: 1) a career-oriented manager is an essential part of career progression, and 2) these managers must take the initiative and time to meet with you to discuss your areas of improvement, put you up for promotions, and defend you to committees, often on their own time; so treat them accordingly.

12 May, 2008

The Science Behind Iron Man

Seems like with every anticipated premier of a 'super hero' genre some dateless journalist working for New Science, Scientific America or the like will write an article about the feasibility of technologies that are leading toward said superheros powers. The premier of IronMan was no exception.


Word to you future writers anticipating writing similar styled articles for upcoming movies....stop it! Comparing face recognition technologies in digital cameras have little to do with targeting weapon systems.

10 May, 2008

Adding HardDrive to VMWare WinXp Virtual Machine

Underestimating the persistent storage of a virtual machine (VM) is pretty standard for me. I generally define my VM hard drive pretty conservatively to avoid wasting filesystem space, adding a new hard drive as required. This is the procedure I go through to add a new hard drive to a WinXp VM.

First, create a new virtual hard drive with VMWare. Begin by editing your VM preferences.



Next, add a new hard drive to your VM.





That completes adding the hard drive to the virtual machine, the next steps are required by the guest OS to get WinXp to recognize the new hard drive.

You first need to start up the disk management application.


You then need to open the c:\WINDOWS\system32\compmgmt property file.
Double-click on the Disk Management selection and a wizard will pop up, follow along with the wizard.










After the partion wizard has completed, exit computer management and you should now have a newly created hard drive on your system.
Viola.

09 May, 2008

Cloning a VMWare Virtual Machine

I've become a recent convert to the virtualization technologies that have been recently developed. I'll postpone the accolades of virtualization for future posts. For now, I'll concentrate on a task I find I regularly perform, but had some trouble locating how to do it.

Virtual machines opens the door for creating a VM template, where the guest OS is installed in a pre-defined manner, and task-specific instances of this template can be created and disposed of when the task is completed. For example, evaluation of a 30-day evaluation software installation is a prime example. Repeated installation, followed by un-installation, on a OS takes its toll leaving remnants of the products in unlikely locations. Creating a new VM instance dedicated to product evaluation and later deleting the entire VM once the product evaluation has completed is...in a word...simple.

This requires defining a VM template, simply put a generic OS installation that you'll clone for specific tasks. This is generated simply by creating a VM and installing the guest OS (documented in VMWare references). What isn't documented is how to clone/copy the VM. That will be the focus of the remainder of this post.

Suppose you have a VM template called WinXpBase, located in /VM/WinXpBase/ and you wish to clone this VM as Clone01 at /VM/Clone01/.

Step 1 - Copy the VM:
$ cp -rf /VM/WinXpBase /VM/Clone01

Step 2 - Update the Virtual Disks

$ cd /VM/Clone01
$ vmware-vdiskmanager -n WinXpBase.vmdk Clone01.vmdk

Step 3 - Rename Files Accordingly
$ mv WinXpBase.vmx Clone01.vmx
$ mv WinXpBase.vmsd Clone01.vmsd

Step 4 - Update File Name References in VM Properties
open the Clone01.vmx file in your favorite editor and search-n-replace instances of WinXpBase with Clone01

Step 5 - Add VM to VM List
As root, edit the following files:
/etc/vmware/vm-list
/etc/vmware/vm-list-private
duplicating the lines with WinXpBase, followed by search-n-replace with Clone01.

Fire up the VM Console and power up your new Clone01 VM.

Last Step - Apply New VM Identifier
Upon applying power to your new Clone01 VM, you'll encounter a pop-up similar to:

This is a one-time pop-up, you should select Create and you've just cloned your very own VM.

07 May, 2008

Parsing File Lists with Tcl

Hey all,
Been a bit of time since my last post. I've been struggling attaining some new information to post, but haven't found anything 'post worthy'. Instead, I wandered back in the ol' memory to find a useful nugget of info that you may find useful.

I love Tcl. From the bottom of my heart, this is the scripting language of choice for this cubicle dweller. A routine use for scripting languages is to batch process vast lists of files. Any well behaved *nix user probably uses camel back file naming notations (e.g. SomeFileForYou.txt), while others may use a underscore delimited notation (e.g. some_file_for_you.txt), but Windows enthusiasts (many of my teammates included) insist on using spaces in their file names (e.g. Some File For You.txt).

Batch processing filenames with spaces can present a problem if you don't parse the list properly. Since Tcl lists are generally seperated by spaces, file names with spaces are parsed as individual files for each word.

Observe:

$ ls /var/tmp/someDir
total 8
-rw-r--r-- 1 fatslowkid fatslowkid 6 2008-05-07 22:38 Some Longer File Name.txt
-rw-r--r-- 1 fatslowkid fatslowkid 7 2008-05-07 22:38 Some Long File Name.txt

$ cat listFiles
1 #!/usr/bin/tclsh
2
3 set fileList [exec find /var/tmp/someDir/]
4 #set fileList [split $fileList "\n"]
5
6 foreach file $fileList {
7 puts '$file'
8 }


Running will result in:
$ ./listFiles
'/var/tmp/someDir/'
'/var/tmp/someDir/Some'
'Long'
'File'
'Name.txt'
'/var/tmp/someDir/Some'
'Longer'
'File'
'Name.txt'


Uncommenting line 4 which splits the list using the newline character as the delimiter solves the problem. Observe:

$ ./listFiles
'/var/tmp/someDir/'
'/var/tmp/someDir/Some Long File Name.txt'
'/var/tmp/someDir/Some Longer File Name.txt'


TaDa. Hope you find this useful, I certainly have over the years.

29 April, 2008

Old School Meets Open Source

I recently mentioned that our division moved office location. As part of the move, we adopted our divisions computer security policies. My employer, a monolithic corporation focusing on government military contracts which they perform at a modest competency.

Given our current decade, it's nearly impossible to ignore the relevance of open-source software. Our company is no exception, and has tried to adopt capitalizing on open-source developments to maintain a competitive edge. For this, I find no fault.

Our recent adoption of our division's computer security policy seems counter productive in the pursuit of utilizing open-source software however. The main handicap is the disallow of visiting blogs of any nature. News flash....many, many, many open-source technical references are published in....wait for it......BLOGS!

The rationale for blocking blogs...to restrict time-wasting websites. Riddle me this caped crusader...if you employ people who spend significant time on non-productive work (wink, wink) why in the name of all that's good are they still working for you! Rather than employ preventions to keep people from doing bad things, just eliminate the people who do said bad things. How damn difficult is that?

23 April, 2008

Avi Frame Decode


fmpeg -i small.avi -r 25/10 -f image2 out%03d.jpg


Yes, if you input AVI is 25fps and you want one of each ten frames then use as output rate 25/10


ffmpeg -i /media/cdrom1/video_ts/vts_01_1.vob -i /media/cdrom1/video_ts/vts_01_2.vob -i /media/cdrom1/video_ts/vts_01_3.vob -vcodec mjpeg -f avi tmp.avi

Makes a motion jpeg video from the 3 unencrypted vobs

20 April, 2008

Breakin' The Law, Breakin' The Law

The imortal words of Rob Halford, front-man of Judas Priest.

Copyright infringements are a delicate matter. As a professional, you should avoid infringements. After all, some day you're own product may find its way to the store shelves and you wouldn't want anyone to steal it without compensating you for it would you?

No, seriously; this post is aimed at identifying a couple products that I have heard (wink, wink) to be quite useful in duplicating production dvds.

Many commercial dvds have some form of copyright protection, restricting many duplication products from working. DVDFab HD Decripter 4 http://www.dvdfab.com isn't such a product. Fact of the matter is that it will work on most dvd's to date. I've heard many people use it to duplicate the dvd contents to their hard drive. The paid version can duplicate the contents to a dvdr, however being a cheapskate I've elected to duplicate it to the hd and used another product to burn it.

DvdShrink http://www.afterdawn.com/software/video_software/dvd_rippers/dvd_shrink.cfm works directly for older dvds, duplicating the contents dvd to dvd. It however is not up to date on many of the improved copyright techniques. So I've heard that duplicating a dvd to hd, and burning the harddrive contents to a dvdr works quite well.

I've heard that these two products combined let you duplicate most of the commercial dvds on the market.

18 April, 2008

Technologies to Minimize Geekhood

Alas my fellow geek our time has come!!

My wife no more than an hour ago underwent Lasik eye correction eliminating her dependency on glasses. She will soon realize all the advantages of those before her with perfect vision. But I ask you.....where was this procedure 20 years ago?

As a boy, my down-spiraling story began in the 4th grade. As a small, skinny kid from the trailer park I had little going for me. My shoes a step down from Converse, my jeans from K-mart, my facial features from centuries of down-beaten Polish/German ancestries, and my hair a step up from collie, or Bovidae. In short, not a lot going for me.

While 'getting the ladies' is not impossible in the 4th grade under those circumstances, throw on a pair of spectacles and you've just signed yourself up to a long decade of virginity...a 10ft boobie free zone following you late into your teens.

Today however we have technologies that even the playing fields. Lasik removes our NF glasses, pec implants giving us a chiseled chest, Alli meds for our belly minimization (read that diet pills), steroids to bulk up, and Viagra to ensure our sexual prowess.

I hesitate to think how my adolescent life would have changed had these been available to me. I probably would have spent every dime on a tackle box full of medications rather than a black beasty hot-rod. I'd have probably got more play with the ladies, and been too tired to study....leading into a long career of "...would you like to supersize?" rather than software engineering.

Eh, maybe it worked out for the better; I loved that car.

16 April, 2008

Software Quality Assurance -- SQA

This posting is aimed at you individuals whose career velocity brought them to a career in SQA:

Your job is a joke....your career a farce...your respect undeserved and absent. No one outside your department respects your job nor opinion.

I say this not to hurt, nor instigate a rebuttal but to point out the large gray elephant in the room, the one no one talks about. Think of me as your mother...tearing off the bandage to let the healing begin.

Some of the nicest people I know work in SQA. That doesn't change my opinion on the matter, it only pushes it to a region where no one talks about it. The fact of the matter is that the majority of the people working in SQA are genuinely nice people, intelligent people, and all around good fellas (in a unisex means). They aren't what I would call motivated people, or driven individuals. No, SQA groupies are typically 9-5'ers....9-3'ers when they can get away with it, 9-5'ers with long lunches when they can't.

I've been involved in professional software engineering going on 11 years now. SQA has had a role in each program that I've been involved in, so I have a working knowledge of their impact on the development process. The fact of the matter is that it has no positive impact on the process. It never has a positive impact. It has yet in 10 years proved to identify a relevant issue or flaw. The little known fact is that SQA is informally responsible for hampering the development process in a form of white-collar unemployment. We literally pay the entire department to push paper, author obsolete processes that don't even casually reflect reality. Only monolithic programs like defense contractors can even afford to employ such wastefulness, sadly because it is your money to burn and not theirs.

So if, my fair reader, you find yourself currently employed in SQA rise above it, grab a firm grip on your bootstraps and give a good yank. Retrain yourself, motivate yourself, do whatever it takes to get out of your career purgatory. You may find a career as an escort (read that as hooker) commands more respect...at least you'll be providing someone a service.

09 April, 2008

Finding Networked Video Cameras

Just seen a vlog entry on Google videos stating how to search for networked video cameras on the Internet.

Simply submit a Google search for:
inurl:/view/index.shtml

You'll find some are password protected....boo, but many will be open...yeah.

Have fun.

06 April, 2008

Vietnam Hippies Revisted

As I drove home from work today I passed a sign in some meatheads yard reading "Support Our Troops - End The War". As a true supporter of our troops I am disappointed and enraged at such displays. I have yet to meet a person who genuinely likes war, probably because I don't spend time with psychos. The fact of the matter is...we are in, and we need to finish the job. Vietnam was a prime example of the moral of our country and our troops when we don't give our troops the support to complete the job.

I'm sure idiots like our yard sign poster would have fit in quite nicely with the hordes of draft-dodgers, drug doping hippies, and the protesters that spit on our troops on their return. I wonder if such meatheads admire the hippies and protesters of yesteryears, or if they believe that they are any different.

These proud soldiers have put their American dreams on hold, and lives on the line to defend those of us that have stayed behind. The absolute least we can do is not embarrass them or stand in the way of allowing them to complete the task at hand and return proud of their accomplishments.

A final note to our yard-poster, Canada is just a quick trip north...I'll help you pack.

31 March, 2008

Off-topic MNDOT Rant

As I found my way to the office this morning while listening to the radio, the radio announcer hinted of a snow forecast for later in the morning. At the time, not a speck of snow, nor rain, was present in brisk morning air.

As I came upon a large vehicle with warning lights....a tow truck? Perhaps an ambulance, or fire engine. What else could it be???

Well, let me tell you. MNDOT in their infinite wisdom decided to salt the road PRIOR to snowfall. What kind of genius salts the road without snow or ice? Surely they are attempting vegetation control of the ditches. That is the only reasonable explanation since applying salt unnecessarily to a road results in cars driving over said salt, crushing said salt, and having the wind blow it off the road into the ditches. Simple as that.

MNDOT isn't underfunded. They have plenty of money to waste, they do it daily. What they don't lack however is incompetence; they surely have a surplus in that area.

24 March, 2008

Same Kid, New Block

Today was the first day at our new office location.

Prior to the move, we found time to scope out our new cube locations.
My immediate neighbor went by the name Andrea. My co-worker taunted
"I'll bet she'll be a smokin' hottie". I laughed, given the hit ratio in
the engineering profession.

Turns out, that he was right. My neighbor is quite fetching. You'd
think that this would be a blessing, given that day-to-day face-to-face
encounters are with people with 5-o'clock shadow....men and women.
Turns out that there is a dark side to having an attractive
neighbor....every man with a pulse finds a reason to visit with her on a daily basis.
This results in an endless stream of distractions so far. While I'm
sure that this is an inconvenience for her, I wouldn't have guessed it
to be such an inconvenience to me (sigh).

15 March, 2008

Burning an Avi/Mpg to Playable Dvd

It took some work, but I finally found out how to burn an Avi or Mpeg file onto a playable dvd.

Step 1: Convert the Avi/Mpeg to proper format
ffmpeg -i InputFile.mpg -aspect 16:9 -target ntsc-dvd foo.mpg

Step 2: Create dvd file structure
dvdauthor -o dvd/ -t foo.mpg
dvdauthor -o dvd/ -T

Step 3: Create ISO Image
mkisofs -dvd-video -v -o DVD.iso dvd/

Step 4: Burn to Disc
growisofs -dvd-compat -Z /dev/scd0=DVD.iso

This will burn an input video file to a playable dvd. This example doesn't include a dvd menu, nor multiple files on one disc. DvdAuthor suffers from the same symptom as many open-source products, little-to-no documentation nor examples. As a result, I wasted about a dozen DVD+R's trying to decrypt it's usage and finally gave up.

Bon Scott Statue Unveiled


I am by all accounts a HUGE AC/DC fan. Have been since the first time my older brother exposed me to "Dirty Deeds Done Dirt Cheap" while racing a Monte Carlo in his 75 Trans Am on the interstate near my home town. Little did I know what an impact that moment would have for the rest of my life.

I became a dedicated fan on that faithful day...and continue to be to-date. That's why it's important to note that 24 February 2008 holds a special day in history. This marks the day that a bronze statue tributing AC/DC first lead man (Dave Evans doesn't count) was unveiled at, his childhood haunt, Fisherman's Warf in Fremantle Australia. It was unveiled as part of the Aussie Rock Concert at Claremont Showgrounds. A proud day for all AC/DC fans.

14 March, 2008

Jeez, We Are Seriously Overpaid

As my current program is coming to a close I've been evaluating alternative programs that my current employer has to offer. In terms of interest, it's pretty slim pickings.

As a result, I've took part in my first phone interview in 3+ years. Things went quite well, the closing question from the HR rep was concerning compensation. Now I'm not currently hitting 6 figures, but I'm charging hard toward the hoop.

Growing up blue collar I know what hard work is; I've shoveled grain, tossed bales, and picked my share of rock for less than $2 / hour. I've spent my share of days working in 90+ degree heat with a gallon of water for the day and little more than a warm bologna and cheese sandwich (oftentimes with mayo.....warmed in the afternoon sun). I've busted hump 'til my back ached, shoulders gave out, and feet grew a crop of blisters from heel to toe. I know what it feels to suffer serious dehydration, and my share of hangovers cured by the recommended dosage of sweating it out with outdoor manual labor. I know what hard work is, and I know what salary you can expect for this type of labor.

I try to keep this in mind time-n-time-again, took keep things in check. I approached an initial salary that competed with my fathers maximum salary he'd been working toward his entire life. Less than 10 years have passed since, and my salary has more than doubled. Rates of change will slow from here on, but still....we are incredibly fortunate in the field that we are in. If you are even moderately competent, you'll demand a salary that will surpass most dual family incomes. Chances are that you'll go home to your well furnished apartment, or quite likely single family home where you'll relax in your easy chair quietly decompressing to the drone of central air. Your reminder of a hard day....a migraine, or perhaps an armful of paperwork/techwork you need to complete before morning. The only joint or muscle issues you'll likely face is a mild case of carpal tunnel.

Just try to remember how easy we have it the next time you find yourself complaining about work. Better yet, keep this in mind on a good day; remembering how well you're paid to do something you truly love.

Cheers.

12 March, 2008

C Storage Classes

In investigating the proper means to declare a global variable, I found myself reviewing some C documentation that I had long forgotten, concerning the Storage Class.

To briefly review, each variable declaration consists of 3 elements: the storage class, the type, and the variable name. For example:


auto int x;
static int y;


The storage class can take the form of auto, extern, static, or register. If not specified, the implicit storage class is that of auto.

The auto storage class is the most frequently used of the classes, primarily because it is the implicit default. Local variables take this form, where the storage is not allocated until the block in which the variable is defined is entered.

The extern storage class specified simply a reference to a variable that is defined elsewhere. Space is therefore not allocated upon encountering this reference, since the storage is allocated elsewhere. This as you may recall is the means to declare a global variable.

The static storage class specifies that the variable cannot be access by functions outside the translation unit in which it was defined. A common error is declaration of a static variable in a header file, which is imported by more than one translation unit not understand that each translation unit essentially created independent copies of the variable. A common practice of declaring a static constant in a header, used by multiple translation units results in multiple copies of the constant, in each translation unit. However, if the static variable simply defines a constant....generally, no-harm-no-foul. It is however worth understanding that it is not a shared reference.

The last storage class is that of register, which notifies the compiler to make the variable as efficient as possible. Ideally, the variable will retain it's location in a cpu register for optimal performance.

11 March, 2008

Global Variables in C

First the disclaimers;
Global variables are bad. Global variables are evil. Global variables are, and will continue to be much like a 6-ft chainsaw, extremely dangerous to use, can regularly be replaced with a safer tool/technique, but are sometimes the proper tool for the job (like cutting down 12-ft diameter trees). None-the-less, on the rare occasion that a global variable is the proper technique, you'll need to know how to define and use them...the subject of the remainder of this post.

The following example will be used to explain techniques for defining and using globals, we will speak to it for the remainder of the post.

package1.h

1 #ifndef PACKAGE1_H
2 #define PACKAGE1_H
3 static const float GlobFloat = 3.14159;
4 static double GlobDouble = 1.23456;
5 extern unsigned GlobUnsigned;
6 //char GlobChar='a';
7
8 void func1(void);
9 #endif

package1.c

1 #include "package1.h"
2 #include
3
4 static int PackageGlobalInt = 1;
5 unsigned GlobUnsigned = 12345;
6
7 void func1() {
8 printf("(%s:%d) PackageGlobalInt(%d)\n",__FILE__,__LINE__,PackageGlobalInt++);
9 printf("(%s:%d) GlobFloat(%f)\n",__FILE__,__LINE__,GlobFloat);
10 printf("(%s:%d) GlobDouble(%lf)\n",__FILE__,__LINE__,GlobDouble++);
11 printf("(%s:%d) GlobUnsigned(%u)\n",__FILE__,__LINE__,GlobUnsigned++);
12 printf("(%s:%d) GlobDouble(%lf)\n",__FILE__,__LINE__,GlobDouble++);
13 printf("(%s:%d) GlobUnsigned(%u)\n",__FILE__,__LINE__,GlobUnsigned++);
14 }
15

package2.h

1 #ifndef PACKAGE2_H
2 #define PACKAGE2_H
3 #include "package1.h"
4
5 extern long GlobLong;
6 void func2(void);
7 #endif

package2.c

1 #include "package2.h"
2 #include
3
4 static int PackageGlobalInt = 2;
5 long GlobLong = 987654321;
6
7 void func2() {
8 printf("(%s:%d) PackageGlobalInt(%d)\n",__FILE__,__LINE__,PackageGlobalInt++);
9 printf("(%s:%d) GlobFloat(%f)\n",__FILE__,__LINE__,GlobFloat);
10 printf("(%s:%d) GlobDouble(%lf)\n",__FILE__,__LINE__,GlobDouble++);
11 printf("(%s:%d) GlobUnsigned(%u)\n",__FILE__,__LINE__,GlobUnsigned++);
12 printf("(%s:%d) GlobLong(%li)\n",__FILE__,__LINE__,GlobLong++);
13 printf("(%s:%d) GlobDouble(%lf)\n",__FILE__,__LINE__,GlobDouble++);
14 printf("(%s:%d) GlobUnsigned(%u)\n",__FILE__,__LINE__,GlobUnsigned++);
15 printf("(%s:%d) GlobLong(%li)\n",__FILE__,__LINE__,GlobLong++);
16 }
17

main.c

1 #include
2 #include "package1.h"
3 #include "package2.h"
4
5 int main() {
6 printf("(%s:%d) main process initializing\n",__FILE__,__LINE__);
7
8 func1();
9 func2();
10
11 printf("(%s:%d) GlobFloat(%f)\n",__FILE__,__LINE__,GlobFloat);
12 printf("(%s:%d) GlobDouble(%lf)\n",__FILE__,__LINE__,GlobDouble);
13 printf("(%s:%d) GlobUnsigned(%u)\n",__FILE__,__LINE__,GlobUnsigned);
14 printf("(%s:%d) GlobLong(%li)\n",__FILE__,__LINE__,GlobLong);
15
16
17 printf("(%s:%d) main process terminating\n",__FILE__,__LINE__);
18 }
19


Globals are a form of interface, so rightly are typically defined in a header file. Package 1 attempts to define 3 global variables. You will soon see that it was only successful in defining one. Before we get into the details, we should review some preliminaries.

#includes
As you should already be aware, header files serve little purpose other than organization of code. When encountering a #include directive, the precompiler does little more than copy-n-pasting the contents into the sourcing file. This is significant in our later discussion.

Static Variables
Static variables cannot be accessed outside a translation unit. For all intents and purposes, a translation unit is a C source file. This, along with what we know about #includes will explain why only one of our intended global variable declarations was successful.

Back To Our Example
The contents of package1.h on lines 3-5 are three attempts to declare a global variable. Note, lines 3-4 define the variables as static, line 5 defining it as external linkage. Why are the first two variables defined as static??? Because if I don't, I get a linker error "multiple definition of . . ." error. Pounding on the keyboard, slowly beating the linker into submission is accomplished by declaring the variable as static. Finally, the linker shuts up; we must have fixed it, right? Wrong!

So, why doesn't lines 3-4 work? More importantly, how do we know it doesn't work? Well, we can confirm that line 4 didn't work by comparing the results of lines package1.c:10, package1.c:12, package2.c:10, package2.c:13, and main.c:12. Note that on package 1 & 2 functions, we are incrementing the contents of GlobDouble. If we were successful in defining a global variable, we should see the variable initially assigned 1.23456, incremented 4 times with a final result of 5.23456.


~/StorageClasses$ ./main | grep GlobDouble
(package1.c:10) GlobDouble(1.234560)
(package1.c:12) GlobDouble(2.234560)
(package2.c:10) GlobDouble(1.234560)
(package2.c:13) GlobDouble(2.234560)
(main.c:12) GlobDouble(1.234560)

Hmmmph, we didn't accomplish what we intended, did we? The static storage class, coupled with the precompiler copy-n-paste behavior resulted in 3 independent variable declarations; one in package1.c, one in package2.c, and one in main.c. Each instance is unique to each translation unit, so incrementing one only affects the declaration in that translation unit.

Line 5 however successfully defined a global variable. How? Well, line 5 simply states that we are declaring a variable, to be defined elsewhere. Declaration simply defines a reference to variable that will be defined elsewhere (in this case, line 5 in package1.c). Translation units: package1.c, package2.c and main.c will each declare a reference to GlobUnsigned, but the single definition of the variable resides in package1.c.

We can confirm that all is well by means of:


~/StorageClasses$ ./main | grep GlobUnsigned
(package1.c:11) GlobUnsigned(12345)
(package1.c:13) GlobUnsigned(12346)
(package2.c:11) GlobUnsigned(12347)
(package2.c:14) GlobUnsigned(12348)
(main.c:13) GlobUnsigned(12349)



It is worth noting that lines 3-4 in package1.h have identical results as line 4 in both package1.c and package2.c. All instances are similar in the matter that their scope is limited to the translation unit in which they are declared.

10 March, 2008

Formal Names For Common C Techniques

If you ever written a C application with custom-defined header files, you've used the pre-processor pattern:


#ifndef FOO_H
#define FOO_H
.
.
.
#endif


Ever wonder what this pattern was called? I never did, until the other day when I searched for it. The pattern is referred to as the 'once-only header' pattern, sometimes called the 'wrapper #ifdef'.

Call it what you want, the intention is to ensure that the header is not copy-n-pasted multiple times in a translation unit, due to multiple direct or indirect includes. The FOO_H is referred to as the 'guard' or 'controlling macro'.

Another common technique that you may have made use of goes something like this:

#ifdef SYS1
#include "sys1.h"
#elif . . .
.
.
.
#endif

This is referred to as a 'computed includes' pattern or technique. Not much else to add, just a name for a common face.

28 January, 2008

I've Been IT'd

Once again I've been IT'd, that is "Information Technology"'d. I have recently began using it as a verb, synonymous with screwed, broken, f**cked...

Is it just me, or is it just our IT department? For the past 3 years, IT has managed to break our systems fairly regularly on a monthly schedule. The past 6 months, they've broken our network a total of 4...count 'em....four times! I'm convinced that 100 monkeys with 100 keyboards couldn't cause any more mayhem and quite possible provide better support.

While they lack competence, they thrive in lack of dedication. Come 3-4 o'clock you'd be hard pressed to fit a sheet of paper between any given meathead that makes up the steady stream of those heading for the door. Jetta after Jetta, off to screw up someone else's machine...or perhaps a marathon session of World of Warcraft (tee hee).

03 January, 2008

I See Windows FS

Ok, sure...it sounded better in my head; a pun on "I see dead people". But once down, I have to admit, it is pretty lame. Too bad! Besides, if you are a reader of this blog you're already aware that I'm humor-impaired.

Being a long-time Linux user I've encountered the daunting decision during an installation....should I make the Windows partition visible from Linux? Most of the time, the answer to this question is a 'hard' _no_. My current computer has Debian installed, and the question was again presented during the installation and I chose, once again, no. Unfortunately, I've recently found a desire to make the partition available for file transfers and manually updated my system to make the partition available. I decided to document the steps of this effort, just for fun.

- Step 1 -- determine the file system identifier for the Windows partition

$ cat /boot/grub/menu.lst
.
.
.
# This entry automatically added by the Debian installer for a non-linux OS
# on /dev/hda1
title Microsoft Windows XP Home Edition
root (hd0,0)
savedefault
makeactive
chainloader +1
.
.
.


Note: a more appropriate means to get the same information would be:

# /sbin/fdisk -l


- Step 2 -- update your fstab


$ cat /etc/fstab
# /etc/fstab: static file system information.
#
#
proc /proc proc defaults 0 0
/dev/hdb1 / ext3 defaults,errors=remount-ro 0 1
/dev/hdb5 none swap sw 0 0
/dev/hdd /media/cdrom0 udf,iso9660 user,noauto 0 0
/dev/hdc /media/cdrom1 udf,iso9660 user,noauto 0 0
/dev/scd0 /media/cdrom2 udf,iso9660 user,noauto 0 0
/dev/fd0 /media/floppy0 auto rw,user,noauto 0 0
/dev/hda1 /mnt/winXp/ ntfs uid=1000,gid=100,umask=0022 0 0


- Step 3 -- create the mount point

# mkdir /mnt/winXp


- Step 4 -- mount the file system

# mount /mnt/winXp


- Step 5 -- pat yourself on the back
(pat, pat)