|
|
Resources: Latest position from APRS My public CVS server My instance of AprsWorld ImageServer Friends: Vanessa Park Jon Sullivan Chris Bell Steve Kehlet Mark Blair Andy Chou |
|
|
 |
||||
|
|
|
|
|
|
|
||||
|
|
|
Login or register: |
|
|
|
||||
|
|
|
New Articles: April 12th, 2008 - Wildflowers and Landmarks My Grandfather's Alfa Romeo Spider March 8th, 2008 - Carrizo Plain 2008 Elections Bridge To Nowhere High Availability October 20th, 2007 - Big Bear Camping October 22nd, 2007 - Fire Scottish Highlands, Aug 7th, 2006 Scottish Highlands, Aug 6th, 2006 August 5th, 2007 - Duck Lake Trail May 26th, 2007 - Kelso Dunes Culloden Battlefield, Aug 5th, 2006 May 20th, 2006 - Irwindale Renaissance Faire Edinburgh, Aug 4th, 2006 The Clifs of Moher, Aug 3rd, 2006 The Burren, Aug 2nd, 2006 Bunratty Castle, Aug 1st, 2006 May 5th, 2007 - Mojave Truck Audio/Data Network 2007 - Master Bath Remodel Centrum: Exit The Ring of Kerry, Jul 31st, 2006 Nikon D200 Victory in 2006! Blarney and Killarney, Jul 30th, 2006 Dublin and Cork, Jul 29th, 2006 Dublin, Jul 27th & 28th, 2006 Married! What Can I Do? April 30th, 2006 - Anza Borrego New desktop: Intel 805 D Macro Photography Jan 7th and 14th, 2006 - Hiking Whiting Ranch Hiking Engaged! Digital Photography with Linux September 5th, 2005 - Living Desert August 19th, 2005 - Mammoth Hiking and Photography July 30th, 2005 - Tucker Wildlife Sanctuary Nikon D70 Death, Fright and Photography Mmmmmm Eggs MythTV Inova T4 May 14th, 2005 - Red Rock April 2nd, 2005 - Death Valley Count Every Vote Act of 2005 Image Archiving Linear Logic ScanGuage Speakeasy DSL Gentoo Linux Ocean Defenders November 6, 2004 - Mojave Super Tuesday, 2004 John Kerry Kayaking Irish Stew ImageServer Ireland, 2004 Canon A80 Camera Jul 25, 2004 - Death Valley Chronic Hiccups May 4th, 2004 Landscaping - My Front Slope Stump Pullin' Yeeee Haw! Feb 22nd, 2004 Feb 16th, 2004 PostgreSQL Logfile Analysis Spam AprsWorld Mountains? Desert? Jan 30th, 2004 Jan 28th, 2004 Encryption Ceiling Cargo Basket Front Bumper Version 2 Asha Exide Orbitals My Teeth Land Rover Valve Jobs Spirits The Matrix: Revolutions Halloween 2003 Greg Davis CDL Linkage Ouzo Democracy Mom's Turkey Gravy Grandma's Guacamole Top Nodes Julian Pie Company DeCSS The KB1DIG 2-meter Halo Antenna Craps Incomming searches Gardening is hard! Geeky dinner party Aug 13th, 2003 SQL and Perl Cancun 2003 Jul 9th, 2003 LongPlayer Aprs intelegence Jun 17th, 2003 Some People's Comments Roof Rack, maybe Dakota is a silly dog The Matrix: Reloaded Chris' Stage Bottle Harness April 23rd, 2003 Cracked Radiator! Vanessa Black Wednesday DVD Burning Under Linux My Satellite Phone Wind! My Near-Death Experience Laser Cannon Revival Front Bumper, Version 2 Open WiFi Network SpamAssassin The Critters Dakota Milton KPC 3 Plus and HTX-252 My AT&T UPS My House Moving, moving moving... Mobile 1 Portola Hills New new house Suse Linux Database images Possible New House In Truck Dr. Pepper My Favorite Toilet Kelso Dunes Desert Trips Ifulmuh Late Thoughts: Dr. Pepper Cooler 265/75-R16 Tires on a DII Linux George W. Bush Rants Driving Reservations Horses Sep 14 2002 Sawdust Bellydancing 2002 Oriental Nights Argus Obsession August 17th Yukon Dives Less notifications My Custom Front Bumper Bracketless, Renamable Links Discreet Winch Rack Temperature Welding Jul 28 2002 Day Trip My COX Cablemodem Jedi Group Revival My Firewall Jedi Group, my T1 and money A Bumperless Discovery! My Custom Rear Bumper Vanessa's 24th Birthday Jun 30th Dive to Long Beach Canyon PHP/PostgreSQL String Quoting Tonsillectomy, Uvulaectomy and Turbinite Reduction Searching functionality 240 Watt CO2 Laser Cannon My Workspace Dr. Pepper The Tulsa Rib Company The quality of hard disks these days Email notification of articles Email notification of comments erikburrows.com source code The Bell Trip to Belize 2002 User Bios User Preferences Login feature Renisance Faire Jun 9, 2002 Computers hate me, and it is mutual. Star Wars Sucks! Horses, Jun 1, 2002 Land Rover Mileage Insomnia, Robin goes evil. 100 Watt Diode Laser Test Firing 1 Amateur Radio The Matrix 2001: A Space Odyssey TDI Deco Class Horses, Apr 30, 2002 APRS Movies Blackbird My Truck Batteries My Truck Vasquez Rocks The Zope Bible PSK31 Mojave Apr-12-2002 100 Watt Diode Laser The New www.erikburrows.com Hunter the Kitty Horses Geeks Yukon May-13-2001 Computers Ironage Jul-4-2001 Programming Languages Jedi Group Ironage Feb-03-2001 Matts Desert Pics Mojave Apr-01-2001 Mojave Jun-09-2001 Mojave Jan-27-2001 Mojave Feb-10-2001 SCUBA |
|
|
|
||||
|
Who is Erik? What's this site about? |
|
| ||
These days, effective digital photograph processing entails several steps, and usually several tools. This is especially true with more advanced or "fringe" equipment or formats, and given the propensity of camera and software manufacturers to support only the top few computer platforms, we Linux users tend to be left behind. Well, in recent years, Open/Free software has been catching up, and we've gotten to the point of being very effective and efficient. It may not be as easy to set up a good workflow on Linux as it is on Windows, but with a little work, and the right tools, we have the capability.
I've done a lot of research on the subject, and for the benefit of everybody's time, I've compiled a list of helpful tools, and I'll explain how I use them. Monitor Color Calibration: The mechanics of downloading/copying the files, processing and converting them, then archiving and printing them is often what people jump to first, but the first step really, is color calibration. After long last, I've finally found a way to calibrate my monitor properly, without installing Windows just to use the couple hundred dollar spider. Nathan Willis wrote an excellent article on monitor calibration, utilizing Norman Koren's color tables. In just a few minutes, almost any monitor can be calibrated, which means what you see on your monitor will be much closer to what your lens saw. This step should get your work flow reasonably well color-calibrated up to the printing step. I have not yet found a way to color calibrate a printer under Linux, though if you use a print lab for your printing, then it's less of an issue. Image Downloading: To get the image files from the camera into your computer, there are three options:
#!/bin/sh
# The name of the new directory, eg 2005-09-15.11:30
dir=`date +'%F.%R'`
# Make the new directory
mkdir $HOME/images/$dir
# Go into the new directory
cd $HOME/images/$dir
# Mount the flash card/camera
mount /dev/sda1 /mnt/flash
# Copy the files
find /mnt/flash/dcim -type f -exec cp -v '{}' . ;
# Unmount the card
umount /mnt/flash
Mounting a flash card, or camera (in storage mode) requires that USB, USB-Storage, and FAT16/FAT32 support be compiled into your kernel. Your flash drive may not be /dev/sda1, if you have other SCSI or generic storage devices attached. Look in your /proc/partitions file for a list of available devices.Many modern Linux distributions, detect and mount flash cards automatically, so the mount command above many not be necessary. The card may just appear within /mnt or /media. Processing, Option #1: Generally, digital cameras produce two types of files, JPEG and RAW. JPEG is a highly compressed format, but is what we call "lossy". It actually throws out some information in order to get the file to compress more. Almost all cameras can produce JPEG files, but to use JPEG files effectively, you will need to experiment some. Your usually have several controls: Size, quality, sharpness, saturation, etc. JPEG files are the smallest (meaning you can fit more on your memory card), and easiest format to deal with, but you will need to experiment with the above settings to see how much quality or size you are willing to lose to get that size advantage. Remember, you can always make your images smaller in the computer, but you can not make them larger again, without losing quality. (Relative to size.) The same goes for sharpness and saturation. You can always sharpen and saturate later, but going backward costs quality. Many cameras, mostly "prosumer" and "pro" cameras also produce RAW files in addition to JPEG (and sometimes other formats). RAW files are completely unprocessed by the camera, and don't compress very well, so the are large, filling up memory cards faster, and taking longer to download and process. However, they have not lost any information, and often give you a little extra latitude in processing before you start losing information. For instance, you can often adjust the exposure a little without washing out shadows or highlights. (Increasing exposure does add noise though, so there's no free lunch here either.) To use RAW files on Linux, you will need a converter tool. There are a few, but the one I use is UFraw. It has a command-line interface, and a Gimp plugin, making it very convenient. UFraw uses the DCraw library, which supports several RAW formats, including Nikon and Canon. After I download my pictures, I use an image viewer like After I've selected a few pictures that are good enough to put the effort into processing, I fire up the Gimp. Gimp stands for Gnu Image Manipulation Program, and is an exellent program, often compared favorably to Photoshop. I've never used Photoshop, and I'm sure it's just great, but I love the Gimp, it's very full featured, and easy to use. JPEG files can be opened directly, RAW files must be opened with the UFraw plugin. When opening a RAW file, the UFraw window pops up, which allows you to adjust some of the image parameters before conversion to 8-bit color for the gimp. My Nikon D70 produces 12-bit RAW files, but almost everybody uses 8 bits per color, as in JPEG files, so a conversion takes place. Obviously, some information gets lost, but by tweaking exposure, white-balance, contrast and saturation, this loss can be minimized. Once you have your image opened in the Gimp, a world of tools are available for tweaking it. My goal is to take the picture correctly while I'm still looking at it, so I try to reprocess it as little as possible. Generally, this means cropping, noise reduction, color tweaking, and sharpening, in that order. Cropping is easy with the "crop" tool. Noise reduction is not always needed, and should be used very sparingly. If your image has noise, in my opinion, you didn't take the picture right. You needed either a lower ISO speed, or a slower shutter speed. Noise means you didn't have enough light. Reducing image noise causes you to lose information, so the trick is to minimize that loss. There are many tools to reduce noise, but my favorite is the Dcam Noise plugin. I find that it is very effective at reducing noise, but maintains image sharpness very well. It can also be used to smooth out people's complexions in distant pictures. Color tweaking is much more challenging, and is something you can take a lifetime to learn, but there are many resources out there. Try GimpGuru or The Gimp Tutorials. I try to use the curves tool as little as possible, but just a touch will help a lot. For instance, the picture above needed just a little help in the high-blues. Last is sharpening. I prefer to do all my sharpening in the post-process, rather than let the camera do it, because as with everything, less is more. Too much sharpening can make the image look weird, or bring out noise. It's best to do it on a big, bright monitor, rather than on the tiny LCD on your camera. The gimp "unsharp mask" filter is the standard. Unsharp mask may sound like it would blur the picture, but what it actually does is apply a "mask" over the parts that are un-sharp, making them sharper. Sharpening with the unsharp mask is kind of like a blunt hammer though, often you want to sharpen just the well-defined edges in your image, and for that you need the smart-sharpener. You can do "smart sharpening" by hand, as described by Eric R. Jeschke here, or you can make your life easier, and use the smart-sharpening plugin. There is one more step that I sometimes employ. It is called "dynamic range extending". This works only for pictures taken in RAW format, and utilizes that wiggle room in exposure. The problem is that digital camera sensors are have a very small "dynamic range". This means that it's hard for them to expose both very bright, and very dark colors at the same time. Often you have to decide which to expose properly, causing you to loose the other. The dynamic range extender plugin, written by Olli Salonen, who also wrote the smart sharpening plugin, takes two images, layered on top of each other, and combines them, usin g the highlights from one, and the shadows from the other, bringing them both into proper exposure. To use it, switch to RAW mode, and take the picture. (Or several, at slightly different exposures. This is called "bracketing".) Open the best looking picture (best on average) in the gimp. Use UFraw's exposure slider to adjust the exposure until the highlights are exposed correctly, and click "open". Next, open the same picture again. Leave all the other parameters the same, and adjust the exposure until the shadows are exposed properly, then click "open". Now cut and paste one picture into a new layer of the other, and run the dynamic range extender tool. You should end up with a very nice looking, high-dynamic-range picture. Processing, Option #2 The above set of tools for image processing, namely the Gimp, UFRaw, and Gimp plugins is an effective, complete system. It does have some drawbacks however. First and foremost, it's not very intuitive. This causes it to take up more time than it should, and feels more like programming than painting. For all of the middle-steps, there is another very capable linux alternative: LightZone. LightZone is not an open-source product, but rather a closed-source, for-profit product produced by Light Crafts Inc. of Palo Alto California. The product installs and runs perfectly on my machine, and is much, much more comfortable to use than the above toolset. The LightZone forums have also proven to be a very useful resource for questions and techniques. I strongly recommend trying out LightZone if you're a Linux using photographer, weather or not you're already comfortable in the Gimp environment. There is a free 30-day trial version you can download, which is fully capable. I'll keep the Gimp around, mostly for the more precise controls of UFraw, superior noise reduction capability of DCam and the Smart Sharpening plugins. LightZone does have both noise reduction and sharpening tools, both of which work very well, so it'll only be for those very special few images I'll need to fire up the Gimp for anymore, and I'm very happy with that. The hours and hours of time you can save by using LightZone will quickly make up for the purchase cost. Printing: The last step is printing. This is the real benefit of digital photography. I often print only about one in every fifty pictures I take, which saves an awful lot of money. Printing at home with an ink-jet printer is fine, but for the best quality, I recommend either a print lab, on-line print lab, or, I've found that the "Sony Picture Station" at your local Kinkos/FedEx produces very high quality prints. For us Linux users, printing at home can be a challenge. Step one is to find the best quality print drivers. For my HP PSC-750, this means the drivers that come with CUPS. The easiest to use, and most powerful tool is gimp-print, but it doesn't support as many printers as CUPS. Gimp-print allows you to adjust almost every print parameter, including layout. If you can, use gimp-print. If you can't, like me, I use Xpp. It has as many adjustments possible as gimp-print, but isn't as easy to use, and isn't integrated into the Gimp. Archiving: Now you have hundreds of pictures, in nice, neatly time stamped directories, and you want to be able to search for pictures from your last birthday, or vacation. You might be able to guess the date and time, but that's kind of a pain, and only works as well as your memory does. You need some way to describe all your images, and search for them later. I use a program called MaPiVi, which I describe using in another article called Image Archiving. Geo-Tagging: 90% of my photos are landscapes or travel pictures, so I often like to reference where the photos were taken. Storing positional information within image metadata is known as geo-tagging. For this purpose, I almost always have a GPS receiver on in my pocket or backpack while I'm out shooting. I take care to synchronize the camera's clock to the GPS time (which is always accurate, as long as it has signal). Then, when I get home, I download my tracklog, and use that to geo-tag my pictures. The tools I use for this are GPSMan to download the tracklog from my Garmin Rino GPS. I save my tracklogs in GPX format, which is a good open-standard XML-based format. I then use Happy Camel to geo-tag my images from the GPX tracklog file. I use Happy Camel becuase it's command-line based, so easily scriptable. The photobook manager programs that come with Gnome and KDE also have geo-tagging functionality. The script I use to invoke Happy Camel is very simple, selecting the most recent tracklog file to use: #!/bin/sh happycamel -v 3 -x -t `ls -1 --sort=time ~/tracklogs/*.gpx | head -n 1` $@This script will take a list of files to geotag, and use the most recent tracklog file to find coordinates. I hope all this helps, if you would like clarification on any of the steps, or have a suggestion or correction, please comment or email me. | ||
|
Comments: nEO (2005-10-16): But it's seem, you don't use any "Color space management"? If you shoot with raw format, you'd better using color space management and ICC profile. But gimp seem will not support color space until gimp-2.4. only lcms can deal with ICC profile now? Erik (2005-10-17): Color space management is a topic that I would like to explore, but you are correct in that Linux applications, and video and print drivers lag behind in this respect. This is something I will keep up on, and if you find useful information, please pass it on! newmikey (2005-10-19): UFRAW can handle color profiles which are applied before gamma adjustment. I use the Bibble profiles in UFRAW and the results are very satisfactory. BTW: Great resource, this site! Erik (2005-10-20): newmikey, For Nikon owners, UFRAW reads the camera's built-in color profile right out of the NIF file. It would be best to have the monitor's color profile, but using the monitor calibration procedure above, you'll get most of the way there. Todd M (2005-10-23): The new Photoshop handles RAW files now. I just got it and it's a great program! I use it to process the RAW images from my Canon EOS 20D. The look awesome! newmikey (2006-06-30): Todd M: do you run Photoshop under Wine (Crossover or other) ? Couzin2000 (2007-09-14): I just read your article, very cool. The existence of LightZone was unknown to me, and I'M sure to try it soon. Maybe I can interest you in a VERY awesome program, which is very resemblant to Photoshop (the later versions, like Creative Suite 2): check out http://www.kanzelsberger.com/. You won't regret it! Pixel is a full-fledged photo editor that handles color profiles, and even though it is a little more expensive than Gimp (35$ right now), it's still lightyears ahead of Photoshop (1000$+). Very much worth the viewing! Erik (2007-09-18): Cousin2000, Pixel does look like a cool product. It's always good to have more tools in the toolbox. My name (2008-01-15): Have you more ideas like that? Erik (2008-01-17): I keep this page updated when I have new ideas. Add a comment: See also: Nikon D70, May 20th, 2006 - Irwindale Renaissance Faire permalink |
| ||
| I just added another project to my public cvs repository: ImageServer. The imageserver project is a PostgreSQL and file based database for images. It allows the single or batch importing of images, automatic reading of image EXIF data, and storage of additional user defined data. It then allows searching based on this data, display and resizing/reformatting of the images. The system has a concept of image "owners", public images, and images accessable only by certain people. Still in development is advanced, typed searching, allowing searches like "newer than March 2004, from Erik, with a Canon a80 camera". Also planned is a SOAP interface, allowing direct utilization of any ImageServer site by another web site or program. You can see the development site at imageserver.erikburrows.com, which is currently serving all the images on this site. Update: 2004/12/02 I just finished the first cut at a reorganization of the ImageServer code and interface. There are still a lot of admin tools to write, but the public side is done. The new front page is faster, and cooler I think, and search results are paged, with pretty buttons. Update: 2005/01/24 Just added to the imageserver front page is something I've wanted to do for a long time: Sequences. Every time I go on a trip, or have a set of images to upload to the imageserver, I set a sequence name. It's a name for the whole group of images. The imageserver now shows the most recent image sequences, as shortcuts to the search tool. Using this new feature, you can see pictures from my camping trip last weekend into the Angeles National Forest. Update: 2005/02/08 As often happens, my interests have shifted to photography, away from updating this site, scuba, kayaking, etc. I'll get back to those, but I need to fully explore photo geekery for a while. To reflect my change in interest, I have added the imageserver "Image of the day" (called "Favorites", since I don't have a favorite every day!) to the navigation-bar of this site. I hope to promote the imageserver site, to get feedback on both the site, and my photographs. I will eventually have a comment system setup on the imageserver site, but until then, please to tell me what you think here. Update: 2005/02/08 I know, I'm an OCD freak. Comments are now active on the imageserver site. Update: 2008/04/14 The Imageserver now uses exiftool instead of ImageMagic to read EXIF codes, and because of this, can interpret GPS coordinate metadata. If the image had coordinate data, the Imageserver view page now links to Google Maps, to point out exactly where the image was taken. This can be seen when you click on this picture: 
Update: 2008/04/21 I removed the front-page "Recent Favorites" section, in favor of a new feature: Galleries. I added a new metadata key, called 'gallery', which allows me to classify the pictures by theme, and better organize those top 1% of pictures. I think this also allows people, myself included, to find a picture "of a nice flower", or "a pretty landscape", etc. | ||
|
Comments: Matt Bell (2005-01-24): Sorry I missed the camping trip, but I did get to go for a glider ride on a beautiful day up in Napa (and drank some excellent wine too!) That looks like a great place to camp, I look forward to the details! Erik (2005-01-24): I've always wanted to go glider flying. When are you coming back down again? Will (2005-02-08): Do you have any software that can read the EXIF information stored in the images? That would be sweet to have the imageserver automatically post that information in the page also. Steve Kehlet (2005-02-08): I like it. Erik (2005-02-08): Steve, thanks! Will, actually, it's already there. If you click on an image, below it is a button called 'Show all Metadata', which will display all the metadata the system has for the image, including any description data I added, as well as the EXIF key-value pairs. Some of it is interesting, but some of the values, like exposure and aperture are in a funky format, which I have not added a converter for. If you click on the image again, you get the full-size original of the picture. Matt Bell (2005-07-11): Hey there, I just got my server up and running and am looking for an easy software package to do photo databases. Would you mind if I play around with ImageServer? Erik (2005-07-11): Of course you can, go to the link near the top-left called "My public CVS Server", and click on the "imageserver" project. There are instructions on how to download it. However, if you really intend to use it yourself, you should give me some time to package it up for general use. If you're interested, I'll do that in the next couple of days. Add a comment: See also: Image Archiving, May 20th, 2006 - Irwindale Renaissance Faire permalink |
| ||
| Chris, Jon Mike and I took for the hills this last Saturday, to try to get some final lens-time with this year's desert wildflower bloom, as well as add some additional California Landmarks to Mike's scrapbook, and give the Alfa a shake-down. First we went to the Antelope Valley California Poppy Reserve by way of Bouquet Canyon Road out of Santa Clarita. The poppies were in full splendor, much more so than when Matt and I went there three years ago. The poppies covered the reserve, and spread miles out in every direction, covering much of the basin. Of course poppies were not the only blooming flower, there were a dozen or more other prominent species. 
After the visit to the reserve, and lunch, we headed out to the Tehachapi mountains in search of landmark plaques and the Tehachapi rail loop: 
| ||
|
Add a comment: permalink |
| ||
Vanessa's Saturn "Moya" and My Truck "Talyn" now have a new sibling: My Grandfather's 1993 Alfa Romeo Spider Veloce, "Lil' Red"!
My Grandfather gave Red to my Mother (of gravy fame), who has lent it to me as a more economical daily-driver than My Truck. After a tune-up and a new cat, it's running great and passes CA's smog test. It's a heck of a lot of fun to drive, and even though I barely fit in it, I really enjoy driving it around! | ||
|
Add a comment: See also: April 12th, 2008 - Wildflowers and Landmarks permalink |