From 17f353d8b803f29d0ace0b77bff48c93f8c96e06 Mon Sep 17 00:00:00 2001 From: Manuel Garrido Date: Sun, 29 Jul 2018 11:49:44 +0200 Subject: [PATCH] Fix Engine init settings This PR fixes a simple bug I found when running the notebook on the engine docker image. Basically, the command: ``` engine = Engine(spark, "/repositories", "standard") ``` Failed with the error: ``` Py4JJavaError: An error occurred while calling o302.count. : org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input Pattern file:/repositories/**/* matches 0 files ``` Which I assume it has to do with the expected structure of the repositories. Replacing that line with: ``` engine = Engine(spark, "/repositories", "siva") ``` Fixed it. --- _examples/notebooks/Example.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_examples/notebooks/Example.ipynb b/_examples/notebooks/Example.ipynb index f4b9a6b..56aa944 100644 --- a/_examples/notebooks/Example.ipynb +++ b/_examples/notebooks/Example.ipynb @@ -21,7 +21,7 @@ ".master(\"local[*]\").appName(\"Examples\")\\\n", ".getOrCreate()\n", "\n", - "engine = Engine(spark, \"/repositories\", \"standard\")\n", + "engine = Engine(spark, \"/repositories\", \"siva\")\n", "\n", "print(\"%d repositories successfully loaded\" % (engine.repositories.count()/2))" ]