## Need Help <Hadoop ProcessBuilder NullPointer Exception >

Moderator: Concepts and Technologies for DS and BDP

AizazZaidee
BASIC-Programmierer
Beiträge: 106
Registriert: 20. Apr 2016 22:49

### Need Help <Hadoop ProcessBuilder NullPointer Exception >

Hello guys,
Anyone of you successfully able to run Hadoop examples on Windows Machine? It fails to get process on Windows Machine but on MAC/Linux it's running fine. Here is the stacktrace

Exception in thread "main" java.lang.NullPointerException
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1012)
at org.apache.hadoop.util.Shell.runCommand(Shell.java:483)
at org.apache.hadoop.util.Shell.run(Shell.java:456)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722) at org.apache.hadoop.util.Shell.execCommand(Shell.java:815) at org.apache.hadoop.util.Shell.execCommand(Shell.java:798) at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:731) at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:489) at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:530) at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:507) at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:305) at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:133) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:144) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
at org.apache.hadoop.mapreduce.Job\$10.run(Job.java:1287)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308)
at de.tud.stg.WordCount.main(WordCount.java:59)

salvaneschi
Moderator
Beiträge: 49
Registriert: 29. Mär 2013 23:51

### Re: Need Help <Hadoop ProcessBuilder NullPointer Exception >

AizazZaidee
BASIC-Programmierer
Beiträge: 106
Registriert: 20. Apr 2016 22:49

### Re: Need Help <Hadoop ProcessBuilder NullPointer Exception >

Thank you professor I have tried this one as well .. here is a more detailed link http://www.codeproject.com/Articles/757 ... s-Platform but seems like it's not working on Windows some how.

I borrowed MAC from a friend and I was able to reach this following output

Jim 6
Joe 2
Kai 2
Lin 4
Sue 2

I am thinking of putting a hashSet so that I can get the desired output but I am not sure as whether this would be a good solution as right now it is running on single Node but when it would run on multiple nodes how would it relate that (1,4) and (4,1) are same? Please guide me how to solve or point out please where I am doing it wrong. I can share code as well.

meichholz
Moderator
Beiträge: 167
Registriert: 30. Mär 2016 08:28

### Re: Need Help <Hadoop ProcessBuilder NullPointer Exception >

Hi,

you can also use a virtual machine (e.g. Virtualbox - https://www.virtualbox.org) to run a Linux environment on your Windows machine.

As you can see from the input data, we assume that every friendship is bidirectional. But just because there are two input entries per pair does not mean that you have to count them twice. Maybe you should think about emitting less key-value pairs. It is not necessary to use a Set.

Best,
Matthias