hadoop - Reading Hive managed table from Java Map Reduce code -


i want read managed hive table data form map reduce job. have managed hive table created table created external hive table. want run map reduce job on final managed hive table. read managed have tables have separator defaults "char 1" ascii character. did this:

public static final string separator_field = new string(new char[] {1}); 

and later did in loop:

end = rowtextobject.find(separator_field, start); 

but when run map reduce jar, illegal argument exception @ above line , line given below:

public void map(longwritable key, text rowtextobject, context context) throws ioexception, interruptedexception 

ps: looked project on github reading managed hive table in mapreduce job, cannot make sense of @ https://github.com/facebook/hive-io-experimental.

suppose have input file below (say xyz.txt):-
111 \001 222
121 \001 222
131 \001 222
141 \001 222
151 \001 222
161 \001 222
171 \001 222
\001 hive default delimiter(say).
now in order parse file loaded hive table using map reduce in map method:-

public class mymapper extends mapper<longwritable, text, text, text>{     public void map(longwritable key, text value,context context) throws java.io.ioexception ,interruptedexception     {          string[]vals=value.tostring().split("\\001");         context.write(new text(vals[0]),new text("1"));      }  } 

your driver method normal 1 follows:-

job.setoutputkeyclass(text.class); job.setoutputvalueclass(text.class); job.setmapperclass(mymapper.class); fileinputformat.addinputpath(job, new path(xyz.txt)); 

so final output follows based on map method have given :-
111 1
121 1
131 1
141 1
151 1
161 1
171 1
it looking parsing doing in map method?


Comments

Popular posts from this blog

javascript - DIV "hiding" when changing dropdown value -

Does Firefox offer AppleScript support to get URL of windows? -

android - How to install packaged app on Firefox for mobile? -