Hive in detail : part 2 (Optimization)

Hive Optimization

The Big Data Trends

1.URL to DB
2.DB driver information
db JDBC driver in hive/lib folder
3.DB username/password.
default logger: log4J (/var/log/hive)
edit /conf/ controls HiveCLI logging.
edit control logging MR tasks.
Hive stores query logs on directory altered at hive-site.xml
using hive.querylog.location property.
Hive data: /tmp/hive-username temp directory
configured with hive.exec.scratchdir variable –> then –>
client tmp: /tmp/hive/username

Operators and Functions

SQL dialect, like,
relational: equal/not
equal operator, greater than, less than, and greater than or
equal, or less than or equal operators
– additional:
check value null or not. LIKE/RLIKE
-arthematic, logical
– [array,maps], struct(.)

– count, sum average.
– array/map –> explode function –> zero/more rows.
hive> SELECT explode(packaged_with) as package_contents FROM
products WHERE prod_num=’98820′;
– array exploded out each array item returned a row. map reduce
job was run.

— hive> DESCRIBE FUNCTION upper;

View original post 653 more words

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s