Category Archives: Uncategorized

Looping over javascript arrays with Prototype framework

Had an array in javascript and tried to loop over it using

for (x in array){

...some code

}

I was confused why it seemed to have lots more members than I had put in with contents that seemed to be functions.  So I used console.log and firebug to print out the array members and saw that they were the methods you could call on an array object.

A quick google revealed that (see here)

using forin on arrays when using Prototype will enumerate all extended methods as well, such as those coming from the Enumerable module, and those Prototype puts in the Array namespace

and that the best (only?) way to do it in Prototype is

myArray.each(function(item) {

// Your code working on item here...

});

It’s all sorted now!

RServe on OSX Snow Leopard

I always seem to have some difficulty getting the Rserve component of the R stats library up and running on OSX.  Installing it via install.packages(“rserve”) from within R  appears to work but when running R CMD Rserve I usually get some sort of error like:

/Library/Frameworks/R.framework/Resources/bin/Rcmd: line 62: exec: Rserve: not found

Compiling from source using the download from http://rosuda.org/Rserve/ never seems to work with all sorts of C related horror.

However, the Rserve component actually was installed but it looks like the paths/links were not quite right.  Typing this:

R CMD /Library/Frameworks/R.framework/Resources/library/Rserve/libs/x86_64/Rserve.so

Got everything up and running. Doing this:

cp /Library/Frameworks/R.framework/Resources/library/Rserve/libs/x86_64/Rserve-bin.so /Library/Frameworks/R.framework/Resources/bin/Rserve

fixes it a bit more permanently

Obscure mysql bug number 1153 (08S01) (my packet’s too big)

Ok, here’s one you don’t see every day.

Importing a mysql dumpfile results in “ERROR 1153 (08S01) at line 265: Got a packet bigger than ‘max_allowed_packet’ bytes”

Errm.

Log in to a mysql console,

enter the magic commands:

set global net_buffer_length=1000000;

set global max_allowed_packet=1000000000;

Modify your import to include the command:

–max_allowed_packet=100M

so you have something like

mysql –max_allowed_packet=100M -u user -p db_name < dump.sql

Try again

Your mileage may vary ( do they say kilometerage elsewhere?) so increase the allowed sizes if you have to.

Taming the Savage Beast (Forums)

We needed a forum for one of our web projects written in rails and a quick google revealed Savage Beast.  It seemed to fit the bill although needed some css tweeks to co-exist with what we currently have.  However, when clicking on its RSS feed icon it threw an ActionController::Request.relative_url_root method missing error which was fixed by creating the class request_error.rb in initializers with the code below:

class ActionController::Request
def relative_url_root
@@relative_url_root ||= case
when @env[“RAILS_RELATIVE_URL_ROOT”]
@env[“RAILS_RELATIVE_URL_ROOT”]
when server_software == ‘apache’
@env[“SCRIPT_NAME”].to_s.sub(/\/dispatch\.(fcgi|rb|cgi)$/, ”)
else

end
end
end

Based on bug reported here: http://railsforum.com/viewtopic.php?pid=73991#p73991.

It all seems to work fine now.

Sending lots of data using Post and Rails (ie. avoid URI too long error)

I was having problems with a link to remote that I had encoded as follows:

link_to_remote ‘Variable’, :url=> sort_variables_surveys_url(:sorted_variables=>sorted_variables, :search_query=>query, :survey_list => years, :sort => key)

The sorted_variables param was far too big in some cases and returning a ‘uri too long’ error.  So I scratched my head for a while and then looking at the link_to_remote docs more closely showed that if I wrapped the ‘sorted_variables’ up inside a div like so:

<div id=”sorted_variables”>

<% sorted_variables.each do |variable| -%>

<%= hidden_field_tag “sorted_variables[]”, variable.id.to_s -%>

<% end -%>

</div>

and then added :submit => ‘sorted_variables’ to the link_to_remote call like this:

link_to_remote ‘Variable’, :submit=> ‘sorted_variables’, :url=> sort_variables_surveys_url(:search_query=>query, :survey_list => years, :sort => key)

then it all seems to work fine.

Getting data out of MySQL (and then back in)

There seems to be lots of ways to get your databases out of a MySQL database and an even more bewildering array of ways to get it back in.  I have found that the following commands seem to be the best way to do it (for me):

Out

mysqldump –databases database_name -u username -p > dumpfile.sql

In

mysql -u username -p database_name < dumpfile.sql

Find the reference here.

15/02/10: Here is another variation on getting stuff out:

mysqldump db_name table1 table2 table3 -u username -p > dumpfile.sql

Zipfiles, CSV and UUIDs

MethodBox currently uses a java servlet to create Comma Separated Value (CSV) files inside a zip file.  These are then sent back to the rails application on demand over http.  What we needed was a metadata file to go into the zip file explaining what the CSV files had in them.  It was easiest to use the active records about the CSV columns to get the metadata.

For the zipfile manipulation thanks go to this post and the original rubyzip API.

So, first make the HTTP request:

http = Net::HTTP.new(“localhost”, 10000)
http.read_timeout=6000
response = http.get(‘csvserver/download/+ filename)

#check that the request was OK
if response.response.class == Net::HTTPOK

#check what the response contains

if response.content_type == ‘application/zip’

#make a new file to hold the zipped data

uuid = UUIDTools::UUID.random_create.to_s

#you might need to install the uuidtools gem

File.makedirs(RAILS_ROOT + “/” + “filestore” + “/” +filename)
uf = File.open(RAILS_ROOT + “/” + “filestore” + “/” + filename  + “/” + uuid+ “.zip”,”w”)

#write the content in the response out to the zip file
uf.write(response.body)
uf.close

After this the metadata is created and written to a string called metadata which can then be added to the zip we just created from the http response.  This next line adds the metadata string to the zip inside a file called metadata.txt.

Zip::ZipFile.open(RAILS_ROOT + “/” + “filestore” + “/” + filename  + “/” + uuid+ “.zip”, Zip::ZipFile::CREATE) {|zip| zip.get_output_stream(“metadata.txt”) { |f| f.puts metadata}}

We can then send the completed file back to the requester

send_file RAILS_ROOT + “/” + “filestore” + “/” + filename  + “/” + uuid+ “.zip”, :filename => “something” + “.zip”, :content_type => @archive.content_type, :disposition => ‘attachment’

I have also been looking at using Ruby to do all the CSV file creation but so far speed is proving to be an issue.  However, I found the following link which mentions several libraries.  I am now looking at using a combination of FasterCSV and CSVScan (also available as a gem).  This enables me to write incredibly complex code like this (don’t you just love ruby’s blocks!):

CSVScan.scan(infile) { |row| line = FCSV.parse(row[0], :col_sep => “\t”); new_row = Array.new; if i==0: variable_hash[key].each {|var| pos.push(line[0].rindex(var))} end; i=i+1; pos.each {|col| val = line[0][col];new_row.push(val)}; csv_arr.push(new_row)}

Yeah, it scared me as well!!  It parses a CSV file for several headers and then pulls out the columns which correspond to these headers and adds them into an array.  In MethodBox these files can have 2000 header (ie columns) and 20000 rows so you can see why speed is important when only pulling out a subset of these.

We can then use FasterCSV to write out the new CSV file:

FasterCSV.open(outfile, “w”) do |csv_file| csv_arr.each {|csv| csv_file << csv} end

More bits == better? AKA Snow Leopard broke my Gems

Snow Leopard install went fine.  Running methodbox after this didn’t.  ImageMagick, libxml and mysql gems were all broken.  Followed this advice to get it all working.  Boils down to re-install MacPorts, install ImageMagick and libxml2 from ports, re-install the RMagick and libxml-ruby gems, install mysql-64bit, re-install mysql gem (in 64 bit mode).

Easy!!  Only took 2 hours.  So that’s about 4 hours (and 2GB+ downloads) for the snow leopard upgrade so far!

Also found this script for installing ImageMagick.  I couldn’t get it to work but maybe you can.

There’s something good around the corner

Up until now we have been using images to fake rounded edges on divs etc.  This is fine until you want to change the styling and resize anything.  So, I had a look at doing it in pure CSS.  Luckily, so have lots of other people eg. Nifty Corners and Spiffy Corners.  I used the Spiffy Corners one since it comes with a nice code creator which saves lots of time and is easy to re-use.  All you have to do is enter in the class name for the CSS, colour of the bar and the background colour and it generates both the CSS and the html.  Job done.