This is one of the sad days (I must have used the superlative here if would have taken into account only the official things) for me after joining Andale (Now vendio). Karthik, the Director of Engg, the guy, "the one" who knows managing people is moving out of Vendio :(
If I remember correctly, it was some time in November 2005, I moved under Karthik from Ravi/Ardaman after completing the Counters MySQL Migration beast. The first project i had worked under him is "Moving the eBay API to the new schema" which went out really good and then started our journey, a journey filled with all the positive adjectives like happiness, joy, love, respect and so on. From there onwards, there is no looking back.
He always tries to find out the interests of us, extends his complete support for them and make sure they gets fulfilled. Back in mid 2006, after working alone on the counters product for almost close to 1.5 years, I expressed my intent to come out of that hell, loneliness. And within no time, I was given a non-Counters project, I started working on a Java project. He is kind enough to give me some time in learning Java also. There are lot of other instances where he proved himself to be "the leader".
Believe it or not, all this has happened through Phone and email. He is in US office and we used to work from Bangalore. (sounds unrealistic...cant do anything..it happened). Luckily, once i got an opportunity to work with him directly for around 4 months. That was an amazing experience, an experience of a life time. The best part in Karthik, he treats you so well, you will never end up with a feeling of hesitation to express something. And he really understands things and try to provide all the help that he can. The other thing that I really like in Karthik is the freedom that he gives you to do things. He always encourages new ideas and always game for experimentation (My guy).
I am just speechless when I heard the word "leaving" from him. I am just shocked/surprised. Cant do much now. He is decided and he is moving. He is the best manager i have worked with., of course, one anyone can work with; i can only say ATB to him for all his future endeavors and I wish i would work with him some time soon.
Sunday, October 21, 2007
Wednesday, June 13, 2007
Key points from June MySQL Meetup (MV chapter)
Vic (CTO, Vendio), John (NOC Engineer, Vendio) and I have attended the Silicon Valley MySQL meet up that happened at Google’s office yesterday. It was organized by Jeremy Cole who is regarded very high in the MySQL community. (I guess most of you might have heard about him before). The session was mostly a Q&A kind of thing where Jeremy answered all the questions. (Lot of questions must be repetitions for him as we (I think some more guys also!!) are attending it for the first time but he was very generous in sharing things)
Below are the things which I found interesting from the session.
a) Question 1: How good is MySQL to store images? (Same as, how good is MySQL for blobs)
Though there is no straight forward answer on how many images (size??) will MySQL hold comfortably, he is saying that we can safely rely on MySQL if the size of blob is in KB’s If the size of blob is in MB’s, then MySQL may not be right choice. One of the main reasons behind this is that MySQL doesn’t use streaming to send blobs across the network. So if there is a blob of size 1G, then it requires 1G (or more?) on server and 1G (or more?) on client too to effectively transfer the blobs. MySQL is trying to address this problem by implementing streaming in its future releases.
For storing big blobs, the other open source tool that we can really look at is MogileFS. This is a very good system for Blobs it seems.
http://danga.com/mogilefs/
b) Question 2: How good is MyISAM compared to INNODB for applications where the data is loaded only once and don't change for ever?
The one thing that MyISAM suffered from all the time is the regular data corruption. But if we have a backup of the data in the file system which we can use at any time to load it back to the DB w/o taking much of time, we can safely use MyISAM. Also one other way to cope up with data corruption easily is, at the time when we are loading the data to Research DB, try to load the same data into a backup MySQL DB and incase of any corruption we can simply override the corrupted files from the backup.
c) Question 3: How effective using Query Cache would be?
There is one very interesting point that Jeremy has made here. Query cache flushes out a query if the table on which that query operates changes. It doesn't matter whether some thing has changed with respect to the rows it has stored or not. This clearly tells that Query Cache is not an option for any application that does decent writes to the DB. It just adds an additional overhead of searching in Query Cache for every query unnecessarily. Please turn off Query Cache or add the SQL_NO_CACHE hint to all the required select queries, if you still have some compelling reasons to go for a Query Cache.
d) Question 4: What's the best way to go about backups?
Jeremy has suggested using LVM snapshots to take backups from INNODB. From his experience, INNODB hot back tool (Commercial tool from INNODB) is not recommended. From my experience, don't go for Zmanda as it is just a script on top of the things offered by MySQL. It is definitely not worth paying 500$ per license every year for this very basic s/w.
e) Question 5: What's the best way to setup MySQL Replication? (Dual-Master or Master-Slave?)
The straight answer is to go for a Dual Master setup. Two other important things he has made is that, always set read-only=1 and skip-slave-mater=1 on master. We have to this once the master comes up. The advantage of doing this is that, if for some reason master fails and we have switched to slave, and when the master comes back again, it should not accept any connections for write. And other very important point he has made here is that, in case of failure on master, never try to apply the lost transactions automatically rather do it manually by comparing the sequence numbers on slave and master. Its better to spend couple of minutes looking at what went wrong rather than spending sleepless nights if the automated script really screws something. :)
f) General Discussion:
a. Size of bin-logs can be around 768M. (Not too high or too low). From his experience, bin-logs of this size would take around 5-10mins for recovery.
b. Always give minimal space for MyISAM (64M ??) as MySQL internally uses MyISAM to store some system tables and also for creating any temp table, it uses MyISAM.
c. The o/p of show status in 4.* versions gives all global values for variables where as in 5.* to get the same o/p we have to use "show global status". "show status" now gives info only for that session.
d. Always set no-auto-rehash under the client section of the my.cnf. This makes sure that MySQL don’t try to create tab completion indexes for every new client connection.
e. There is a way to look at the life cycle of a query on MySQL. This works from 5.0.37 community and 5.0.42 enterprise onwards. Please go through the below link for extra information on this, http://dev.mysql.com/tech-resources/articles/using-new-query-profiler.html .
f. Always try to keep number of files in a directory in hundreds and not more than that as many FS’s including ext3 do a linear search of files inside a directory.
I am really impressed by the way Jeremy has handled the whole session and I am looking for the future events.
Hmm..Have to go back to work. :)
Below are the things which I found interesting from the session.
a) Question 1: How good is MySQL to store images? (Same as, how good is MySQL for blobs)
Though there is no straight forward answer on how many images (size??) will MySQL hold comfortably, he is saying that we can safely rely on MySQL if the size of blob is in KB’s If the size of blob is in MB’s, then MySQL may not be right choice. One of the main reasons behind this is that MySQL doesn’t use streaming to send blobs across the network. So if there is a blob of size 1G, then it requires 1G (or more?) on server and 1G (or more?) on client too to effectively transfer the blobs. MySQL is trying to address this problem by implementing streaming in its future releases.
For storing big blobs, the other open source tool that we can really look at is MogileFS. This is a very good system for Blobs it seems.
http://danga.com/mogilefs/
b) Question 2: How good is MyISAM compared to INNODB for applications where the data is loaded only once and don't change for ever?
The one thing that MyISAM suffered from all the time is the regular data corruption. But if we have a backup of the data in the file system which we can use at any time to load it back to the DB w/o taking much of time, we can safely use MyISAM. Also one other way to cope up with data corruption easily is, at the time when we are loading the data to Research DB, try to load the same data into a backup MySQL DB and incase of any corruption we can simply override the corrupted files from the backup.
c) Question 3: How effective using Query Cache would be?
There is one very interesting point that Jeremy has made here. Query cache flushes out a query if the table on which that query operates changes. It doesn't matter whether some thing has changed with respect to the rows it has stored or not. This clearly tells that Query Cache is not an option for any application that does decent writes to the DB. It just adds an additional overhead of searching in Query Cache for every query unnecessarily. Please turn off Query Cache or add the SQL_NO_CACHE hint to all the required select queries, if you still have some compelling reasons to go for a Query Cache.
d) Question 4: What's the best way to go about backups?
Jeremy has suggested using LVM snapshots to take backups from INNODB. From his experience, INNODB hot back tool (Commercial tool from INNODB) is not recommended. From my experience, don't go for Zmanda as it is just a script on top of the things offered by MySQL. It is definitely not worth paying 500$ per license every year for this very basic s/w.
e) Question 5: What's the best way to setup MySQL Replication? (Dual-Master or Master-Slave?)
The straight answer is to go for a Dual Master setup. Two other important things he has made is that, always set read-only=1 and skip-slave-mater=1 on master. We have to this once the master comes up. The advantage of doing this is that, if for some reason master fails and we have switched to slave, and when the master comes back again, it should not accept any connections for write. And other very important point he has made here is that, in case of failure on master, never try to apply the lost transactions automatically rather do it manually by comparing the sequence numbers on slave and master. Its better to spend couple of minutes looking at what went wrong rather than spending sleepless nights if the automated script really screws something. :)
f) General Discussion:
a. Size of bin-logs can be around 768M. (Not too high or too low). From his experience, bin-logs of this size would take around 5-10mins for recovery.
b. Always give minimal space for MyISAM (64M ??) as MySQL internally uses MyISAM to store some system tables and also for creating any temp table, it uses MyISAM.
c. The o/p of show status in 4.* versions gives all global values for variables where as in 5.* to get the same o/p we have to use "show global status". "show status" now gives info only for that session.
d. Always set no-auto-rehash under the client section of the my.cnf. This makes sure that MySQL don’t try to create tab completion indexes for every new client connection.
e. There is a way to look at the life cycle of a query on MySQL. This works from 5.0.37 community and 5.0.42 enterprise onwards. Please go through the below link for extra information on this, http://dev.mysql.com/tech-resources/articles/using-new-query-profiler.html .
f. Always try to keep number of files in a directory in hundreds and not more than that as many FS’s including ext3 do a linear search of files inside a directory.
I am really impressed by the way Jeremy has handled the whole session and I am looking for the future events.
Hmm..Have to go back to work. :)
Monday, January 8, 2007
icicibank.com -- Make online banking a nightmare
icicibank.com --> Unless i am forced to use this site for all my monetary transactions, i must not have even looked at this site. This is undoubtedly the worse internet banking site. Whenever you try to do a transaction, more often than not, it will be in maintainance mode. In general, maintainance will be done once in a while. Incase of icicibank.com, it is completely the opposite.
In their quest to provide the best service, they are keeping the site under maintainance 24x7. :))) . Guys, agreed, you are the best one in town. please come out of the maintainance mode now atleast.. :))
Customers --> Don't only think of the sops they provide. It also matters how well they provide the services to you. And w/o any doubt, icicibank response to user queries is a complete mess. You have to wait indefinitely for your call to get transferred to the right department. God knows, whether it gets transferred finally or not. You have to be a really lucky person to get your query answered.
Adding to all these, icici also holds the record for loosing the crictical DMAT documents in the MUMBAI rain waters last year. Great!! i am hearing the applause too :) .. And people are asked to fill their application forms again.. :) This explains the carelessness of these people towards customers.
Summary --> ICICI service sucks, icicibank.com is hopeless, icicidirect may get drowned in rain waters sometime. Never choose icici as your salary account.
In their quest to provide the best service, they are keeping the site under maintainance 24x7. :))) . Guys, agreed, you are the best one in town. please come out of the maintainance mode now atleast.. :))
Customers --> Don't only think of the sops they provide. It also matters how well they provide the services to you. And w/o any doubt, icicibank response to user queries is a complete mess. You have to wait indefinitely for your call to get transferred to the right department. God knows, whether it gets transferred finally or not. You have to be a really lucky person to get your query answered.
Adding to all these, icici also holds the record for loosing the crictical DMAT documents in the MUMBAI rain waters last year. Great!! i am hearing the applause too :) .. And people are asked to fill their application forms again.. :) This explains the carelessness of these people towards customers.
Summary --> ICICI service sucks, icicibank.com is hopeless, icicidirect may get drowned in rain waters sometime. Never choose icici as your salary account.
Sequences in MySQL
Auto_increment serves as a parallel to Sequences in MySQL.
But auto_increment has some limitations compared to its counterpart in Oracle.
But auto_increment has some limitations compared to its counterpart in Oracle.
- AUTO_INCREMENT is limited to one column per table
- AUTO_INCREMENT must be assigned to a specific table.column (not allowing multi table use)
- AUTO_INCREMENT is INSERTed as a not specified column, or a value of NULL
Even if a row is deleted, the sequence wont be reset to its previous value.
Prior Connect in MySQL
MySQL used to support CONNECT BY PRIOR in its previous version(s). But due to some of the buggy code they have found in recursion and its associated modules, they have turned off this in the new versions.
The work arounds for this problem can be found at this place.
http://forums.mysql.com/read.php?10,32818,32818#msg-32818
http://jan.kneschke.de/projects/mysql/sp/
A very good theoritical explanation for the same can be found at
http://www.intelligententerprise.com/001020/celko.jhtml?_requestid=697912
The work arounds for this problem can be found at this place.
http://forums.mysql.com/read.php?10,32818,32818#msg-32818
http://jan.kneschke.de/projects/mysql/sp/
A very good theoritical explanation for the same can be found at
http://www.intelligententerprise.com/001020/celko.jhtml?_requestid=697912
Subscribe to:
Posts (Atom)
