본문 바로가기

Bigdata 동영상

HBase basic commands and RegionSplitter [하둡 동영상 강의]

이곳에는 간단한 HBase 명령어 실행과 RegionSplitter 사용을 본다. 

동영상 제작해서 만들어 업로드기까지는 시간이 엄청나게 걸리는 작업인 걸 이번에 알았다.........




[hadoop@h001 hbase]$ ./bin/hbase shell

HBase Shell; enter 'help<RETURN>' for list of supported commands.

Type "exit<RETURN>" to leave the HBase Shell

Version 0.94.7, r1471806, Wed Apr 24 18:48:26 PDT 2013



hbase(main):001:0> help

HBase Shell, version 0.94.7, r1471806, Wed Apr 24 18:48:26 PDT 2013

Type 'help "COMMAND"', (e.g. 'help "get"' -- the quotes are necessary) for help on a specific command.

Commands are grouped. Type 'help "COMMAND_GROUP"', (e.g. 'help "general"') for help on a command group.


COMMAND GROUPS:

  Group name: general

  Commands: status, version, whoami


  Group name: ddl

  Commands: alter, alter_async, alter_status, create, describe, disable, disable_all, drop, drop_all, enable, enable_all, exists, is_disabled, is_enabled, list, show_filters


  Group name: dml

  Commands: count, delete, deleteall, get, get_counter, incr, put, scan, truncate


  Group name: tools

  Commands: assign, balance_switch, balancer, close_region, compact, flush, hlog_roll, major_compact, move, split, unassign, zk_dump


  Group name: replication

  Commands: add_peer, disable_peer, enable_peer, list_peers, remove_peer, start_replication, stop_replication


  Group name: snapshot

  Commands: clone_snapshot, delete_snapshot, list_snapshots, restore_snapshot, snapshot


  Group name: security

  Commands: grant, revoke, user_permission


SHELL USAGE:

Quote all names in HBase Shell such as table and column names.  Commas delimit

command parameters.  Type <RETURN> after entering a command to run it.

Dictionaries of configuration used in the creation and alteration of tables are

Ruby Hashes. They look like this:


  {'key1' => 'value1', 'key2' => 'value2', ...}


and are opened and closed with curley-braces.  Key/values are delimited by the

'=>' character combination.  Usually keys are predefined constants such as

NAME, VERSIONS, COMPRESSION, etc.  Constants do not need to be quoted.  Type

'Object.constants' to see a (messy) list of all constants in the environment.


If you are using binary keys or values and need to enter them in the shell, use

double-quote'd hexadecimal representation. For example:


  hbase> get 't1', "key\x03\x3f\xcd"

  hbase> get 't1', "key\003\023\011"

  hbase> put 't1', "test\xef\xff", 'f1:', "\x01\x33\x40"


The HBase shell is the (J)Ruby IRB with the above HBase-specific commands added.

For more on the HBase Shell, see http://hbase.apache.org/docs/current/book.html


hbase(main):002:0> help 'create'

Create table; pass table name, a dictionary of specifications per column family, and optionally a dictionary of table configuration.
Dictionaries are described below in the GENERAL NOTES section.
Examples:

  hbase> create 't1', {NAME => 'f1', VERSIONS => 5}
  hbase> create 't1', {NAME => 'f1'}, {NAME => 'f2'}, {NAME => 'f3'}
  hbase> # The above in shorthand would be the following:
  hbase> create 't1', 'f1', 'f2', 'f3'
  hbase> create 't1', {NAME => 'f1', VERSIONS => 1, TTL => 2592000, BLOCKCACHE => true}
  hbase> create 't1', 'f1', {SPLITS => ['10', '20', '30', '40']}
  hbase> create 't1', 'f1', {SPLITS_FILE => 'splits.txt'}
  hbase> # Optionally pre-split the table into NUMREGIONS, using
  hbase> # SPLITALGO ("HexStringSplit", "UniformSplit" or classname)
  hbase> create 't1', 'f1', {NUMREGIONS => 15, SPLITALGO => 'HexStringSplit'}

hbase(main):003:0> 

hbase(main):039:0> create 'ta1', 'cf1'

$ ./bin/hbase shell

hbase(main):035:0> describe 'ta1'

$ ./bin/hbase shell

DESCRIPTION                                          ENABLED                    

 'ta1', {NAME => 'cf1', DATA_BLOCK_ENCODING => 'NONE false                      

 ', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0',                            

  VERSIONS => '3', COMPRESSION => 'NONE', MIN_VERSIO                            

 NS => '0', TTL => '2147483647', KEEP_DELETED_CELLS                             

 => 'false', BLOCKSIZE => '65536', IN_MEMORY => 'fal                            

 se', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'                            

 }                                                                              

1 row(s) in 0.0660 seconds



hbase(main):039:0> disable 'ta1'
hbase(main):036:0> is_enabled 'ta1'

false                                                                           

0 row(s) in 0.0160 seconds



hbase(main):037:0> enable 'ta1'

0 row(s) in 2.1680 seconds



hbase(main):038:0> is_enabled 'ta1'

true                                                                            

0 row(s) in 0.0170 seconds

hbase(main):044:0> drop 'ta1'


ERROR: Table ta1 is enabled. Disable it first.'


Here is some help for this command:

Drop the named table. Table must first be disabled: e.g. "hbase> drop 't1'"




hbase(main):045:0> disable 'ta1'

0 row(s) in 2.1170 seconds



hbase(main):046:0> drop 'ta1'

0 row(s) in 1.1630 seconds


==> drop 시  hdfs 에 생성되어 있던 /hbase/ta1  디렉토리가 삭제 처리됨


hbase(main):047:0> list

TABLE                                                                           

hbase009                                                                        

regionsplit_table                                                               

test                                                                            

textxx                                                                          

4 row(s) in 0.0570 seconds



hbase(main):048:0> create 'table01', 'cf'
0 row(s) in 1.1100 seconds


hbase(main):049:0> put 'table01', 'row001', 'cf:a', 'value i wanna put'
0 row(s) in 0.1260 seconds


hbase(main):050:0> put 'table01', 'row001', 'cf:b', 'value b i wanna put'
0 row(s) in 0.0200 seconds


hbase(main):051:0> put 'table01', 'row001', 'cf:c', 'value c i wanna put'
0 row(s) in 0.0150 seconds


hbase(main):052:0> put 'table01', 'row002', 'cf:2a', 'value 2a i wanna put'
0 row(s) in 0.0120 seconds


hbase(main):053:0> put 'table01', 'row003', 'cf:3a', 'value 3a i wanna put'
0 row(s) in 0.0260 seconds


hbase(main):054:0> scan 'table01'

ROW                   COLUMN+CELL                                               
 row001               column=cf:a, timestamp=1372602441690, value=value i wanna  put                                                       
 row001               column=cf:b, timestamp=1372602450824, value=value b i wanna  put                                                     
 row001               column=cf:c, timestamp=1372602456583, value=value c i wanna put                                                     
 row002               column=cf:2a, timestamp=1372602470758, value=value 2a i wanna put                                                   
 row003               column=cf:3a, timestamp=1372602481567, value=value 3a i wanna put                                                   
3 row(s) in 0.1360 seconds


hbase(main):054:0> scan 'table01'

ROW                               COLUMN+CELL                                                                                    
 row001                           column=cf:a, timestamp=1372602441690, value=value i wanna put                                  
 row001                           column=cf:b, timestamp=1372602450824, value=value b i wanna put                                
 row001                           column=cf:c, timestamp=1372602456583, value=value c i wanna put                                
 row002                           column=cf:2a, timestamp=1372602470758, value=value 2a i wanna put                              
 row003                           column=cf:3a, timestamp=1372602481567, value=value 3a i wanna put                              
3 row(s) in 0.0500 seconds

hbase(main):067:0> get 'table01', 'row001'


COLUMN                            CELL                                                                                           
 cf:a                             timestamp=1372602441690, value=value i wanna put                                               
 cf:b                             timestamp=1372602450824, value=value b i wanna put                                             
 cf:c                             timestamp=1372602456583, value=value c i wanna put                                             
3 row(s) in 0.0240 seconds

hbase(main):068:0> get 'table01', 'row001', 'cf:a'


COLUMN                            CELL                                                                                           
 cf:a                             timestamp=1372602441690, value=value i wanna put                                               
1 row(s) in 0.0220 seconds

hbase(main):070:0> get 'table01', 'row001', 'cf:a', 'cf:b'


COLUMN                            CELL                                                                                           
 cf:a                             timestamp=1372602441690, value=value i wanna put                                               
 cf:b                             timestamp=1372602450824, value=value b i wanna put                                             
2 row(s) in 0.0270 seconds

hbase(main):071:0> get 'table01', 'row001', ['cf:a', 'cf:b']

COLUMN                            CELL                                                                                           
 cf:a                             timestamp=1372602441690, value=value i wanna put                                               
 cf:b                             timestamp=1372602450824, value=value b i wanna put                                             
2 row(s) in 0.0330 seconds

hbase(main):072:0> 

hbase(main):071:0> get 'table01', 'row001', ['cf:a', 'cf:b']

hbase(main):074:0> scan 'table01'

ROW                               COLUMN+CELL                                                                                    

 row001                           column=cf:a, timestamp=1372602441690, value=value i wanna put                                  

 row001                           column=cf:b, timestamp=1372602450824, value=value b i wanna put                                

 row001                           column=cf:c, timestamp=1372602456583, value=value c i wanna put                                

 row002                           column=cf:2a, timestamp=1372602470758, value=value 2a i wanna put                              

 row003                           column=cf:3a, timestamp=1372602481567, value=value 3a i wanna put                              

3 row(s) in 0.0630 seconds

hbase(main):075:0> import java.util.Date 

=> Java::JavaUtil::Date


hbase(main):076:0> Date.new(1372602456583).toString()

=> "Sun Jun 30 23:27:36 KST 2013"




$ hbase org.apache.hadoop.hbase.util.RegionSplitter 
                    test_table HexStringSplit -c 3 -f f1


cf.


[hadoop@h001 hbase]$ ps auxk -rss|less

[hadoop@h001 hbase]$ jmap -heap 
[hadoop@h001 hbase]$ lsof -uhadoop | wc -l
[hadoop@h001 hbase]$ ps -o pid, comm,user, thcount -u hadoop



[refered to http://hortonworks.com/blog/apache-hbase-region-splitting-and-merging/]

[refered to this HBase book below]