Jason Martin’s Software and Design

So, I have written like a million versions of these in different languages, then I found out that Ruby has an in built CSV parser, so I said cool. I read through the Pickaxe, got the info, plugged it in and boom, no worky, I hate it when things no worky. So I spent the next hour on google looking for how to fix it, people seem to not be having the problems I am having, that is, it just doesn’t work…weird, but fuck it, I have written enough of them to do it my damn self, all I wanted was to parse a csv and maintain the column headers so that I could dynamically create activerecord objects in my rails app and not have to worryabout enforcing column order, or parsing it in the controller. It took me about 20 minutes to write, and yes it shows it, heres the kicker, it works.

That is, like out of the box, plug and play, whatever, it works for me, so maybe it will work for you, that is, unless you want to stick with the poorly documented and wholly mystical ruby version, you might be better off, but I like doing things myself, that way, I completely understand how it works, why, and if I want to make a change I can.

Anyway, so here it is:

Using a csv like this for the example:

"Song Name","Artist","Album","Duration"
"Girls And Boys","Good Charlotte","The Young and the Hopeless",03:05
"Meet Virginia","Train","Unknown",03:59
"Sacrifice","Elton John","Unknown",05:06

class Kcsv
	def initialize(file, options)
		@file = file
		@options = options
		options[:header] == true ? @headers = parse_header_line : @headers = false
		@children = []
		parse
	end
 
	def parse
		i = 0
		@file.rewind
 
		@file.each_line do |line|
			#looping through each line of the file
			if i > 0 then # we are past the first line
				x = 0 #we set the index for our position in the headers.
				row = {} #the row
				if not line.include? ',' then #Here we check to see if it is a single column
					row[@headers[x][0]] = clean(line) if not @headers[x].nil?
					row[x] = clean(line) if @headers[x].nil?
				else #we have multiple columns
					line.split(',').each do |column|
						#now we need to know if we have headers
						if !@headers.nil? then
								row[@headers[x][0]] = clean(column) if not @headers[x].nil?
						else
							row[x] = clean(column)
						end
						x = x + 1
					end
				end
				@children << row
			else
				i = i + 1 #just there to skipp the first line
			end
		end
		return true
	end
 
	def to_a
		@children
	end
 
	protected
	def clean(string)
		return string.gsub('"','').strip
	end
	def parse_header_line
		headers = []
		accepted_headers = ["ebay_id","image","price","quantity","title","desc","id","action","is_live","is_type"]
		@file.rewind
		@file.each_line do |line|
			x = 0
			if not line.include? ',' then
				headers << [clean(line),x] if accepted_headers.include? clean(line)
				headers << [x,x] if not accepted_headers.include? clean(line)
			else
				line.split(',').each do |col|
					headers << [clean(col).strip,x]
					x = x + 1
				end
			end
			break
		end
		return headers
	end
end
#this is just to test that it works as expected, returning and array of arrays with to_a
csv = Kcsv.new(File.open('songs.csv','r'), :header =&gt; true)
csv.to_a.each do |row|
	puts "#{row["Song Name"]} by #{row["Artist"]} from #{row["Album"]} -- #{row["Duration"]}"
end

Anyway, there it is, have some fun. There maybe a better, or faster way, however, this is working, so I am happy enough.

Hey All

Been awhile, I know. While working on one of my projects, it came up that we would need to import thousands of ebay items into the shop, and be able to export them using the ebay file exchange, which dumps out a .csv file. It seems .csv parsers are the most common task I perform, I have written like a million for rails and php over the years, so I slapped something together quickly to do it and oops, I didn’t notice that ebay lists the item’s price with a comma. Ack, Comma Separated File, and there is a comma in one of the values, it’s easy to forget, but can really mess you up. Well, since I was using rails which means ruby, this is not such a problem, I can fix that easy enough with some regex.

To make a long story short, and since the price field was and will only forever be the offending field, I got away clean with a simple quick fix:

fields = line.gsub(/£(\d+),(\d)/,"\\1\\2").split(',')

Got rid of that offending , and the £ sign to boot since we save currency amounts as floats, the £ sign is independent of that, and we use number_to_currency for that. Simple I know, but man, what an effing pain sometimes.

Well, gotta get back to relearning Rails, with the release of 2.0, there’s so much new stuff that it’s driving me nuts. Personally, I don’t see 2.0 as an improvement, except for the caching stuff, and the sexy migrations, those are cool…but that’s it.

A nice screenshot from my current project, as always, I am cursed with creating intranet like apps, which aren’t publicly accessible. This one is from a project integrating a Rails database management system with eBay using their trading API. The whole thing is built from scratch in Rails.

So far it features automatic thumbnail generation and image sorting into directories, c39 and EAN barcode generation, and a connection to eBay that allows stock to be automatically listed.

When it’s finished, I might just create a publicly accessible version just for future client to peruse.