Skip to content

Smart People write Smart Code

Everyone can write code with little understanding of programming and logic. However, writing smart code is never an easy job and requires lots of efforts and thoughts. I had compiled a list of guideline around two years back, about how to write smart code? If anyone can follow and implement these guidelines i am quite sure they can get top positions in all quality coding competitions.

  1. Write as simple code as you can.
  2. Write as less code as you can.
  3. Write as much documentation as you can.
  4. Write as many test cases as you can.
  5. Wrap your code with try/catch, and display useful messages to user.
  6. Log all the possible errors, warnings and useful messages.
  7. Never give too much responsibility to any class or method
  8. Code should always be reviewed by more than one person.
  9. Always think about the environment in which your code will run and take care of all possible errors of that environment.
Advertisements

MySQL 5.1 Installation on Linux

MySQL installation on linux is very easy and only few minutes game. However, it has few little complications if you haven’t done it before. In this post i will explain how to install the MySQL on a linux box for a production use. To make mysql stable for production use you have to take care of few extra steps as mentioned in the post.

Most linux distribution comes with MySQL version of 5.0.x. If you want to install MySQL 5.1 you would first have to uninstall the old version.

Uninstall MySQL 5.0

First get the list of intalled mysql packages with the following command:

rpm -qa | grep -i mysql

or

rpm -qa | grep -i MySQL

It will show you all the installed modules. Now uninstall them one by one using:

rpm -e mysql

This could give you error because of dependencies. If you got error than use the following command to remove mysql without those dependencies:

rpm -e mysql –nodeps

Download MySQL 5.1 binaries

Now you have successfully uninstalled mysql. Download the new version (in our case 5.1) you want to install from http://dev.mysql.com/downloads/mysql/

You need to download the following modules:

  • MySQL server
  • MySQL devel
  • MySQL shared
  • MySQL client

You can use wget command to download these files.

Installing MySQL 5.1

Now install them one by one:

rpm -ivh MySQL-server-5.1.45-1.glibc23.x86_64.rpm
rpm -ivh MySQL-devel-5.1.45-1.glibc23.x86_64.rpm
rpm -ivh MySQL-shared-5.1.45-1.glibc23.x86_64.rpm
rpm -ivh MySQL-client-5.1.45-1.glibc23.x86_64.rpm

Now you have installed all the modules of MySQL. Test the service status:

service mysql status

MySQL running (9993)                                       [  OK  ]

Make it Secure

The default mysql installation has some security loop holes such as empty root password, anonymous users, test database, etc. You need to remove all of these if you are going to use MySQL in production environment.

Run the following script:

/usr/bin/mysql_secure_installation

It will ask you different steps to remove all the security loop holes. Set the strict root password, remove remote access for root, remove anonymous users, and remove test database.

Verify mysql installation:

mysql -V

mysql  Ver 14.14 Distrib 5.1.45, for unknown-linux-gnu (x86_64) using readline 5.1

MySQL Configuration for Production Database

Now lets configure the mysql for the optimum production environment:

vi /etc/my.cnf

[mysqld]
socket=/var/lib/mysql/mysql.sock
datadir=/var/lib/mysql/data

By default mysql is installed under /var/lib/mysql and all the database created directly inside /var/lib/mysql. However, i prefer to use another folder under mysql named data for all the databases of mysql. That’s why i have set my datadir=/var/lib/mysql/data. If you want to use the same setting you will have to follow the following steps:

  1. stop mysql using service mysql stop
  2. create a new folder named data under /var/lib/mysql
  3. move all the files from /var/lib/mysql to /var/lib/mysql/data
  4. start mysql using service mysql start

It will be good if you enable binary logs on your server from start as you might need them for replication.

#replication master-server configuration
#server-id=1
#relay-log=/var/lib/mysql/logs/mysql-relay-bin
#relay-log-index=/var/lib/mysql/logs/mysql-relay-bin.index
log-error=/var/lib/mysql/logs/mysql.err
#master-info-file=/var/lib/mysql/logs/mysql-master.info
#relay-log-info-file=/var/lib/mysql/logs/mysql-relay-log.info
log-bin=/var/lib/mysql/logs/bin-logs
binlog-do-db=vbossdb
#binlog-ignore-db=
# end replication

Once you have set the above in your my.cnf you need to create logs directory under /var/lib/mysql/logs for binary and other logs of mysql.

Make sure mysql owns both data and mysql directories, you can do so using the following:

chown /var/lib/mysql/data mysql
chgrp /var/lib/mysql/data mysql
chown /var/lib/mysql/logs mysql
chgrp /var/lib/mysql/logs mysql

If most of your database is based on MyISM and you need best optimal performance for your MyISM tables use the following configuration. But make sure you change the values according to the RAM available on your system. I have 8GB RAM available on my server and want to use 3GB for my MyISAM tables as i have very large MyISAM tables with mysql partitions.

MySQL Replication Made Easy

MySQL Replication can be done in many ways such as master-master, master-slave, etc. Today i will talk about Master-Slave replication in which one MySQL instance serves as Master database and second instance serve as the secondary database.

Master-Slave replication can be used in many ways, following are few advantages of using Master-Slave MySQL replication

  1. Master server will be used to perform all the CRUD operation while slave can be used to run heavy queries, thus distributing the server load.
  2. Slave pull all the information from server and serves as the mirror of master so its a real-time backup of your database.
  3. You can test your applications with the production data and without connecting to your master/primary database, connect it to slave/secondary database instead.

Lets start with the setup of Master instance.

Setting Master MySQL Instance

Use the following steps to setup your Master MySQL instance.

  1. Stop mysql server if it is running
  2. Edit the my.cnf and enter the following replication properties
  3. Start the mysql server instance
  4. Read the mysql.err file to see if there was any error during startup
  5. Get master status and note down the required properties
  6. Create replication user

Master Server Properties

#replication master-server configuration

server-id=1

relay-log=/var/lib/mysql/logs/mysql-relay-bin

relay-log-index=/var/lib/mysql/logs/mysql-relay-bin.index

log-error=/var/lib/mysql/logs/mysql.err

master-info-file=/var/lib/mysql/logs/mysql-master.info

relay-log-info-file=/var/lib/mysql/logs/mysql-relay-log.info

log-bin=/var/lib/mysql/logs/bin-logs

binlog-do-db=bisdb

binlog-ignore-db=test

# end replication

Following table explains all of these properties:

Once you have updated your my.cnf file with the above properties. You are done with the master configurations, restart mysql server instance and view the error.log file which you specified in the above properties. See, if there is any error in the error.log or not. I am assuming you did everything right and there was not any error.

Following are few useful notes to read before we move onto next step.

Note: If you omit server-id (or set it explicitly to its default value of 0), a master refuses connections from all slaves.

Note: For the greatest possible durability and consistency in a replication setup using InnoDB with transactions, you should use innodb_flush_log_at_trx_commit=1 and sync_binlog=1 in the master my.cnf file.

Note: Ensure that the skip-networking option is not enabled on your replication master. If networking has been disabled, your slave will not able to communicate with the master and replication will fail.

Now login to mysql console.

mysql> show server status;

+—————–+———–+————–+——————+

| File            | Position  | Binlog_Do_DB | Binlog_Ignore_DB |

+—————–+———–+————–+——————+

| bin-logs.000003 | 106 | bisdb        | test             |

+—————–+———–+————–+——————+

1 row in set (0.00 sec)

As you can see that bin-log file is bin-logs.000003 and position is 106, write down both of these values as you will need them to configure slave box. (Use the values that comes on your box)

You will need to create a user with the replication privileges and that user will be used to pull the data from master box.

mysql> create user slave_user;

mysql> grant replication slave on *.* to slave_user@’slave-box-ip’ identified by ‘slaveuser’;

Okay, Now you are done with the master configuration, time to configure your slave box. (Don’t forget to use some strict password for slave user)

Setting Slave MySQL Instance

Use the following steps to setup your Slave MySQL instance.

  1. Stop mysql server if it is running
  2. Edit the my.cnf and enter the following replication properties
  3. Start the mysql server instance
  4. Read the mysql.err file to see if there was any error during startup
  5. Set the binary log position on slave
  6. Check slave status

Slave Server Properties

#replication slave-server configuration

server-id=2

master-host=192.168.168.8

master-user=slave_user

master-password=slaveuser

master-connect-retry=60

replicate-do-db=bisdb

replicate-ignore-db=test

log-error=/var/lib/mysql/logs/mysql.err

relay-log=/var/lib/mysql/logs/mysql-relay-bin

relay-log-index=/var/lib/mysql/logs/mysql-relay-bin.index

slave_exec_mode=IDEMPOTENT

slave-skip-errors=1062,1060,1050,1396,1146

# end replication

Most of the properties are self explanatory so i am going to skip them except last two. 1) slave_exec_mode is used to supress the duplicate key errors. 2) slave-skip-errors is used to continue replication when these errors occur. By default mysql replication stops when there is any error during the replication process. If the error occured contained in the given list of error codes then replication will continue otherwise will halt.

Now restart mysql server and view the mysql.err file to see if there is any error in your configuration or it is started normally. Lets assume All is Well! and login to mysql to set the binary log position.

mysql> CHANGE MASTER TO MASTER_HOST=’192.168.168.8′, MASTER_USER=’slave_user’, MASTER_PASSWORD=’slaveuser’, MASTER_LOG_FILE=’bin-logs.000003′, MASTER_LOG_POS=106;

mysql> show slave status\G

It would show you that its connecting to master for reading the new events. You can test your replication by making some changes in the database which you used in replicate-do-db and any changes made in this database on master box should be replicated to the slave box. Whereas any change made in the ignore-db should not be replicated.

Conclusion

Mysql replication is a very good feature and can be used in many scenarios and the most common ones are backup, performance, etc. In this article i talked about the basic replication which you will use when you are setting up a new environment. But in case of running mysql server instance you might want to make few more changes in the replication process to copy the master data to the slave box.

EST to EDT or CST to CDT Timezone Changes

Today i’ve noticed an interesting behavior in one of our application. We have soft switch running in EST timezone and we got the CDRs with date i.e ‘2010-03-14 02:03:44’. Our application is written in Java and we are using JODA date time library. When we try to enter that date in database our application got data truncation error why? because EST timezone has changed at ‘2010-03-14 02’ and it became ‘2010-03-14 03’ in EDT.

That means timestamp with a value of ‘2010-03-14 02:03:44’ is an invalid timestamp and JODA and MySQL both ignored that timestamp and raised errors.

The trick we did to resolve this error is we added one hour in our timestamp so that time which was before applying the change was ‘2010-03-14 02:03:44′, becomes ”2010-03-14 03:03:44’ after the change. And, it is a valid time in EDT so both JODA and MySQL didn’t raise any error and bug got fixed.

Garbage Collection

by jeremymanson

A little while ago, I got asked a question about when an object is allowed to be collected. It turns out that objects can be collected sooner than you think. In this entry, I’ll talk a little about that.

When we were formulating the memory model, this question came up with finalizers. Finalizers run in separate threads (usually they run in a dedicated finalizer thread). As a result, we had to worry about memory model effects. The basic question we had to answer was, what writes are the finalizers guaranteed to see? (If that doesn’t sound like an interesting question, you should either go read my blog entry on volatiles or admit to yourself that this is not a blog in which you have much interest).

Let’s start with a mini-puzzler. A brief digression: I’m calling it a mini-puzzler because in general, for puzzlers, if you actually run them, you will get weird behavior. In this case, you probably won’t see the weird behavior. But the weird behavior is perfectly legal Java behavior. That’s the problem with multithreading and the memory model — you really never know what the results of a program will be from doing something as distasteful as actually running it.

Anyway, suppose you have a class that looks like this:


class FinalizableObject {
  int i;  // set in the constructor
  int j; // set by the setter, below
  static int k;  // set by direct access
  public FinalizableObject(int i) {
    this.i = i;
  }
  public void setJ(int j) {
    this.j = j;
  }
  public void finalize() {
     System.out.println(i + " " + j + " " + k);
  }
}

And then you use this class, thus:


void f() {
  FinalizableObject fo = new FinalizableObject(1);
  fo.setJ(2);
  FinalizableObject.k = 3;
}

Let’s say that by some miracle the finalizer actually runs (Rule 1 of why you don’t use finalizers: they are not guaranteed to run in a timely fashion, or, in fact, at all). What do you think the program is guaranteed to print?

Those of you who are used to reading these entries will realize immediately that unless they actually already know the answer, they have no idea. Let’s try to reason it out, then.

First, we notice that the object reference fo is live on the stack when all three variables are set. So, the object shouldn’t get garbage collected, right? The finalizer should print out 1 2 3, yes?

Would I have asked if that were the answer?

It turns out that the VM, as usual, is going to play some tricks here. In the words of the JLS, optimizing transformations of a program can be designed that reduce the number of objects that are reachable to be less than those which would naively be considered reachable. What this means is that the VM is going to make your object garbage sooner than you think.

The VM can do a few things to effect this (yes, this is the correct spelling of effect). First, it can notice that the
object is never used after the call to setJ, and null out the reference to fo immediately after that. It’s reasonably clear that if the finalizer ran immediately after that, you would see 1 2 0.

That’s not the end of it, though. The VM can notice that:

  • This thread isn’t using the value written by that write to j, and
  • There is no evidence that synchronization will make this write visible to another thread.

The VM can then decide that the write to j is redundant, and eliminate it that write altogether. Woosh! You get 1 0 0.

At this point, you are probably expecting me to say that you can also get 0 0 0, because the programmer isn’t actually using the write to i, either. As a matter of fact, I’m not going to say that. It turns out that the end of an object’s constructor happens-before the execution of its finalize method. In practice, what this means is that any writes that occur in the constructor must be finished and visible to any reads of the same variable in the finalizer, just as if those variables were volatile. This paragraph originally read incorrectly

The immediate question is, how does the programmer avoid this insanity? The answer is: don’t use finalization!

Okay, that’s not enough of an answer. Sometimes you need to use finalization. There’s a hint several paragraphs up. The finalizer takes place in a separate thread. It turns out that what you need to do is — exactly what you would do to make the code thread-safe. Let’s do that, and look at the code again.


class FinalizableObject {
  static final Object lockObject = new Object();
  int i;  // set in the constructor
  int j; // set by the setter, below
  static int k;  // set by direct access
  public FinalizableObject(int i) {
    this.i = i;
  }
  public void setJ(int j) {
    this.j = j;
  }
  public void finalize() {
     synchronized (lockObject) {
       System.out.println(i + " " + j + " " + k);
     }
  }
}

And then you use this class, thus:


void f() {
  synchronized (lockObject) {
    FinalizableObject fo = new FinalizableObject(1);
    fo.setJ(2);
    FinalizableObject.k = 3;
  }
}

The finalizer is now guaranteed not to execute until all of the fields are set. When that sucker runs, you will see 1 2 3.

Oddly, I’ve been writing for almost an hour and I haven’t gotten to my coworker’s question yet. In the interests of brevity, I’ll make this a series. More later.

Memory Management – Java

http://www.ibm.com/developerworks/websphere/techjournal/0909_supauth/0909_supauth.html

Multi-Core Programming In Java

http://www.javaworld.com/javaworld/jw-09-2007/jw-09-multicoreprocessing.html