hdfs - Spark Streaming program to process a file with more than 10 lines -


i need solve problem program should process files have lines greater 10.

below code:

val sparkconf = new sparkconf().setappname("spooldirspark") val ssc = new streamingcontext(sparkconf, seconds(2))  val lines = ssc   .filestream[longwritable, text, textinputformat]("hdfs:///loclahost:8020/avin")   .map{ case (x, y) => (x.tostring, y.tostring) }  val data = lines.count()   ssc.start() ssc.awaittermination() 


Comments

Popular posts from this blog

ubuntu - PHP script to find files of certain extensions in a directory, returns populated array when run in browser, but empty array when run from terminal -

php - How can i create a user dashboard -

javascript - How to detect toggling of the fullscreen-toolbar in jQuery Mobile? -