Hive in detail : part 2 (Optimization)

Hive Optimization

The Big Data Trends

hivesite.xml
property:
1.URL to DB
2.DB driver information
db JDBC driver in hive/lib folder
3.DB username/password.
——————————
default logger: log4J (/var/log/hive)
edit /conf/hive-log4j.properties controls HiveCLI logging.
edit hive-exec-log4j.properties control logging MR tasks.
Hive stores query logs on directory altered at hive-site.xml
using hive.querylog.location property.
————————————
Hive data: /tmp/hive-username temp directory
configured with hive.exec.scratchdir variable –> then –>
client tmp: /tmp/hive/username

————————————
Operators and Functions
————————————

SQL dialect, like,
relational: equal/not
equal operator, greater than, less than, and greater than or
equal, or less than or equal operators
– additional:
check value null or not. LIKE/RLIKE
-arthematic, logical
– [array,maps], struct(.)

———————————–
functions
———————————
– count, sum average.
– array/map –> explode function –> zero/more rows.
hive> SELECT explode(packaged_with) as package_contents FROM
products WHERE prod_num=’98820′;
– array exploded out each array item returned a row. map reduce
job was run.

check SHOW FUNCTIONS, DESCRIBE FUNCTIONS.
— hive> DESCRIBE FUNCTION upper;
—…

View original post 653 more words

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s