본문 바로가기

Hadoop Ecosystem

RHadoop execution




> b.time <- proc.time()

> small.ints= to.dfs(1:900000)


13/06/27 23:26:25 INFO util.NativeCodeLoader: Loaded the native-hadoop library

13/06/27 23:26:25 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library

13/06/27 23:26:25 INFO compress.CodecPool: Got brand-new compressor

Warning message: In to.dfs(1:9e+05) : Converting to.dfs argument to keyval with a NULL key


> result = mapreduce(input = small.ints, map = function(k,v) cbind(v,v^2))


Warning: $HADOOP_HOME is deprecated.


packageJobJar: [/tmp/RtmpMyf0SJ/rmr-local-enve035c0c399e, /tmp/RtmpMyf0SJ/rmr-global-enve03545a5d0b, /tmp/RtmpMyf0SJ/rmr-streaming-mape0368e92da0, /home/hadoop/hadoop/hadoop-data/hadoop-unjar4602773027836836610/] [] /tmp/streamjob3298825833921073946.jar tmpDir=null

13/06/27 23:26:48 INFO mapred.FileInputFormat: Total input paths to process : 1

13/06/27 23:26:49 INFO streaming.StreamJob: getLocalDirs(): [/home/hadoop/hadoop/hadoop-data//mapred/local]

13/06/27 23:26:49 INFO streaming.StreamJob: Running job: job_201306271950_0010

13/06/27 23:26:49 INFO streaming.StreamJob: To kill this job, run:

13/06/27 23:26:49 INFO streaming.StreamJob: /home/hadoop/hadoop/libexec/../bin/hadoop job  -Dmapred.job.tracker=h001:9001 -kill job_201306271950_0010

13/06/27 23:26:49 INFO streaming.StreamJob: Tracking URL: http://h001:50030/jobdetails.jsp?jobid=job_201306271950_0010

13/06/27 23:26:50 INFO streaming.StreamJob:  map 0%  reduce 0%

13/06/27 23:27:37 INFO streaming.StreamJob:  map 3%  reduce 0%

13/06/27 23:27:44 INFO streaming.StreamJob:  map 7%  reduce 0%

13/06/27 23:27:53 INFO streaming.StreamJob:  map 20%  reduce 0%

13/06/27 23:27:59 INFO streaming.StreamJob:  map 37%  reduce 0%

13/06/27 23:28:02 INFO streaming.StreamJob:  map 63%  reduce 0%

13/06/27 23:28:08 INFO streaming.StreamJob:  map 77%  reduce 0%

13/06/27 23:28:12 INFO streaming.StreamJob:  map 90%  reduce 0%

13/06/27 23:28:15 INFO streaming.StreamJob:  map 100%  reduce 0%

13/06/27 23:28:49 INFO streaming.StreamJob:  map 100%  reduce 100%

13/06/27 23:28:49 INFO streaming.StreamJob: Job complete: job_201306271950_0010

13/06/27 23:28:49 INFO streaming.StreamJob: Output: /tmp/RtmpMyf0SJ/filee031c1ae183

Warning: $HADOOP_HOME is deprecated.


Warning: $HADOOP_HOME is deprecated.


Deleted hdfs://h001:9000/tmp/RtmpMyf0SJ/filee0356fbd3da


> proc.time() - b.time

   user  system elapsed 

 62.190   3.884 219.894