Author Archives: sangroyaamit

About sangroyaamit

I am a PhD student in computer science at INRIA Grenoble.

Installing EC2 CLI tools

Setting Up the Amazon EC2 CLI Tools on RHEL, Ubuntu, or Mac OS X You must complete the following setup tasks before you can use the Amazon EC2 CLI tools on your own computer. Topics Download and Install the CLI … Continue reading

Posted in Uncategorized | Leave a comment

Stages in Hive

A Hive job consists of one or more stages , with dependencies between different stages. As you might expect, more complex queries will usually involve more stages and more stages usually requires more processing time to complete. A stage could … Continue reading

Posted in Uncategorized | Leave a comment

Significant Parameters in Hive

hive.join.cache.size Default Value: 25000 Added In: How many rows in the joining tables (except the streaming table) should be cached in memory. hive.map.aggr Default Value: true Added In: Whether to use map-side aggregation in Hive Group By queries. mapred.reduce.tasks Default … Continue reading

Posted in Uncategorized | Leave a comment

kill hadoop job and child processes

hadoop job -kill <my_job_id> Use pkill -f, which matches the pattern for any part of the command line pkill -f my_pattern pkill -f Child

Posted in Uncategorized | Leave a comment

OpenNebulla (IaaS) – Documentation

homepage: http://opennebula.org/start blog: http://blog.opennebula.org/ Grid 5000 deploy and install: an old page: https://www.grid5000.fr/mediawiki/index.php/Deploying_and_Using_IaaS_Clouds_on_Grid’5000 new updated: https://www.grid5000.fr/mediawiki/index.php/Deployment_Scripts_for_IaaS_Clouds_on_Grid%275000 for a beginner: https://www.grid5000.fr/mediawiki/index.php/IaaS_and_PaaS_Clouds_on_Grid5000 To test: https://www.grid5000.fr/mediawiki/index.php/IaaSclouds_OpenNebula_UserGuide Opennebula vs Openstack http://blog.opennebula.org/?p=4042 http://blog.opennebula.org/?p=4372 http://alax.me/post/22080094990/choosing-a-cloud-platform-my-impressions http://www.linkedin.com/groups/OpenStack-vs-Eucalyptus-vs-OpenNebula-2685473.S.54382975 a master thesis (focus on Eucalyptus): https://docs.google.com/viewer?url=http://web.it.kth.se/~maguire/DEGREE-PROJECT-REPORTS/101118-Victor_Delgado-with-cover.pdf

Posted in Uncategorized | Leave a comment

Linux Tips

Set PATH PATH=/usr:/bin/:usr/local/bin:. This is a very important environment variable. This sets the path that the shell would be looking at when it has to execute any program. It would search in all the directories that are present in the above … Continue reading

Posted in Uncategorized | Leave a comment

Some useful scripts

Include multiple jar files and run a java program export JAR_HOME=/usr/local/hadoop-1.0.0 export JAR_LIB_HOME=/usr/local/hadoop-1.0.0/lib for f in $JAR_HOME/*.jar do JAR_CLASSPATH=$JAR_CLASSPATH:$f done for g in $JAR_LIB_HOME/*.jar do JAR_CLASSPATH=$JAR_CLASSPATH:$g done   export JAR_CLASSPATH #the next line will print the JAR_CLASSPATH to the shell. … Continue reading

Posted in Uncategorized | Leave a comment