Monthly Archives: January 2015

Installing EC2 CLI tools

Setting Up the Amazon EC2 CLI Tools on RHEL, Ubuntu, or Mac OS X You must complete the following setup tasks before you can use the Amazon EC2 CLI tools on your own computer. Topics Download and Install the CLI … Continue reading

Posted in Uncategorized | Leave a comment

Stages in Hive

A Hive job consists of one or more stages , with dependencies between different stages. As you might expect, more complex queries will usually involve more stages and more stages usually requires more processing time to complete. A stage could … Continue reading

Posted in Uncategorized | Leave a comment

Significant Parameters in Hive

hive.join.cache.size Default Value: 25000 Added In: How many rows in the joining tables (except the streaming table) should be cached in memory. hive.map.aggr Default Value: true Added In: Whether to use map-side aggregation in Hive Group By queries. mapred.reduce.tasks Default … Continue reading

Posted in Uncategorized | Leave a comment

kill hadoop job and child processes

hadoop job -kill <my_job_id> Use pkill -f, which matches the pattern for any part of the command line pkill -f my_pattern pkill -f Child

Posted in Uncategorized | Leave a comment