25 June 2008

Green my apple

Rails people have a cult for Apple and I have to say Iphone looks cool :-)
It will be cooler when Apple will remove the Iphone components which can harm Nature (and humans are part of Nature ...)

I support strongly Greenpeace for its campaigns and I have been voluteering now for 2 years in eco-union
a non-for-profit ecological association.

So if you work for a company or public administration involved in sustainabe development and you need a web site, you know who to call.

16 June 2008

Banging my head with some rake error messages

It's the end of a long day. The perfect timing to waste time on trivial issues ... I completly agree with John Nunemaker that http://railstips.org/2008/6/10/programmers-should-give-up-more-often

This one took 30 minutes instead of 2 minutes if my brain had been at 100% !

rake -T --trace
(in /home/jeanmichel/ruby/projects/myproject/myproject1.0)
rake aborted!
uninitialized constant ActionMailer
/usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:2237:in `const_missing'
/usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require'
/usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:27:in `require'
/usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require'
/usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:27:in `require'
/home/jeanmichel/ruby/projects/myproject/myproject1.0/vendor/rails/railties/lib/tasks/rails.rb:7:in `load'
/home/jeanmichel/ruby/projects/myproject/myproject1.0/vendor/rails/railties/lib/tasks/rails.rb:7:in `each'
/usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require'
/usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:27:in `require'
/usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:2149:in `load'

/usr/bin/rake:19:in `load'

This was because my rake task requires a lib/newsletters_mailer.rb file which was had a
require ''newsletter_mailer

importing a file app/models/newsletter_mailer.rb (typical class NewsletterMailer < ActionMailer::Base)

Don't ask me why I had this 'require' in place!!! Rails automatically imports models with all its magic.

12 June 2008

Do no set svn global-ignores for schema.rb

In the subversion configuration file ~/.subversion/config , you can set some global-ignores to a set of whitespace-delimited globs which Subversion will ignore in its 'status' output, and while importing or adding files and directories.

Mine was:
global-ignores = ._* *.log *.html-gzip-* ruby_sess.* *.rej .DS_Store schema.rb

It happens that the Rails framework has a file called vendor/rails/activerecord/lib/active_record/schema.rb ...

OOOPS! This file was not committed which broke the script/console ...

Better removing 'schema.rb' from my global-ignores!

Also worth checking the /etc/subversion/config file

11 June 2008

mysql command lines basic

mysql -u root
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 21
Server version: 5.0.45-Debian_1ubuntu3.3 Debian etch distribution

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

use myapplicationl_development;

Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
SELECT * FROM mailing_jobs where status != 'done';

Empty set (0.00 sec)

mysql> show databases;

mysql> show tables;

# Use back quotes for tables whose names contain spaces (very bad idea if you ask me)
select count(*) from `gmail account`;

Capistrano deploy fails because of svn problem

If you come accross:

/yourdomain2.0$ cap deploy
* executing `deploy'
* executing `deploy:update'
** transaction: start
* executing `deploy:update_code'
updating the cached checkout on all servers
* executing "if [ -d /home/yourdomain/public_html/beta.yourdomain.com/shared/cached-copy ]; then svn update -q -r65 /home/yourdomain/public_html/beta.yourdomain.com/shared/cached-copy; else svn checkout -q -r65 svn+ssh://yourdomain@dev.yourdomain.com/home/yourdomain/svn/yourdomain2.0/trunk /home/yourdomain/public_html/beta.yourdomain.com/shared/cached-copy; fi"
servers: ["beta.yourdomain.com"]
[beta.yourdomain.com] executing command
** [err] Permission denied, please try again.
** [err] Received disconnect from 666.999.999.999: 2: Too many authentication failures for yourdomain
** [err] svn: Connection closed unexpectedly
command finished
*** [deploy:update_code] rolling back
* executing "rm -rf /home/yourdomain/public_html/beta.yourdomain.com/releases/20080610143536; true"
servers: ["beta.yourdomain.com"]
[beta.yourdomain.com] executing command
command finished
command "if [ -d /home/yourdomain/public_html/beta.yourdomain.com/shared/cached-copy ]; then svn update -q -r65 /home/yourdomain/public_html/beta.yourdomain.com/shared/cached-copy; else svn checkout -q -r65 svn+ssh://yourdomain@dev.yourdomain.com/home/yourdomain/svn/yourdomain2.0/trunk /home/yourdomain/public_html/beta.yourdomain.com/shared/cached-copy; fi" failed on beta.yourdomain.com

The solution is quick and easy, add the following line to your capistrano deploy.rb:

default_run_options[:pty] = true

Thanks Mr capistrano Google group!

Filling up my root partition with a table of 3.7Gb

Here we go again!

Yesterday, I filled twice my linux partitions! The first time because I had mysql configure to use binlog and the second time because the Rails development log was filled with the details of the SQL query!!!

This time, the size of the data for the 50 000 emails filled the /var/lib/mysql/lecool_development/emails.MYD file which contains the data of the "emails" table.

I used the following command to find the big files:

sudo find / -xdev -size +104857600c
And it found:
-rw-rw---- 1 mysql mysql 3942782004 2008-06-10 19:58 emails.MYD

I deleted with a SQL "DELETE FROM emails" the data and that was it ...

Hum hum, time to upgrade to a new hard drive with bigger partitions and configure mysql to store its data into the home partition ... or buy a mac??? Let's see in 2009 ;-)

10 June 2008

Filling up my root partition with mysql logs

I was trying to optimize a script on Rails to send 50 000 emails (TODO Add link) and I came across a problem of space with my root partition:

The script crashed with an error message explaining that the hard drive was full.

Using commands like

du -s * | sort -nr | head

or the slower Ubuntu "Disk Usage Analyser", it turned out that the /var/log/mysq was full of huge logs!!!

/var/log/mysql$ ll
total 1925788
-rw-rw---- 1 mysql adm 98 2007-04-10 10:57 mysql-bin.000558
-rw-rw---- 1 mysql adm 117 2008-06-01 15:01 mysql-bin.001456
-rw-rw---- 1 mysql adm 117 2008-06-01 21:45 mysql-bin.001457
-rw-rw---- 1 mysql adm 141 2008-06-02 09:45 mysql-bin.001458
-rw-rw---- 1 mysql adm 104940360 2008-06-10 16:24 mysql-bin.001468
-rw-rw---- 1 mysql adm 104940040 2008-06-10 16:25 mysql-bin.001469
-rw-rw---- 1 mysql adm 104941440 2008-06-10 16:25 mysql-bin.001470
... etc

As a quick fix, I just deleted manually these files with
sudo rm mysql-bin.00148*

A more permanent solution, found on the web, was to edit:
sudo gedit /etc/mysql/my.cnf

Comment all references to log_bin:
# log_bin = /var/log/mysql/mysql-bin.log
# WARNING: Using expire_logs_days without bin_log crashes the server! See README.Debian!
# expire_logs_days = 10
# max_binlog_size = 100M
#binlog_do_db = include_database_name
#binlog_ignore_db = include_database_name
sudo /etc/init.d/mysql restart

Obviously, I will have to refactor a bit the code of ar_mailer to take in account the fact that I am sending 50 000 identical newsletters to 50 000 people.
ar_mailer uses a "emails" table with a "mail" column which is the email content. In my case, the "mail" is duplicated 50000 times + it's a 340KB HTML email. No wonder it filed my sql logs!

I just found out about Evan Weaver top-secret-tuned-mysql-configurations-for-rails post, I really loved the PS ;-)
# PS. Do not under any circumstances enable binlog

Next time, I have to configure mysql, I know what I'll use ...