amazon web services - How to use output of RedShift query as input of an EMR job? -


so limited understand of redshift plan going problem...

i want take results of query, , use them input emr job. best way go programmaticly.

currently emr job takes flat file s3 input, , use amazon java sdk, set job , everything.

should write output of redshift query s3, , point emr job there, , remove file after emr job has completed?

or redshift , aws skd offer more resourceful way directly pipe query redshift emr, cutting out the s3 step?

thanks

recently spoke memebers of amazon redshift team, said solution in works.

this pretty easy - no need sqoop. add cascading lingual step @ front of job executes redshift unload command s3:

unload ('select_statement') 's3://object_path_prefix' [ ] credentials [as] 'aws_access_credentials'  [ option [ ... ] ]

then can either process export directly on s3, or add s3distcp step bring data onto hdfs first.

this lot more performant adding sqoop, , lot simpler maintain.


Comments

Post a Comment

Popular posts from this blog

php - Calling a template part from a post -

i include iframe within post in wordpress. located inside content of post, meaning cannot in post template file, otherwise either before or after. i'd add facebook plugin within posts: <div id="socialwrap"><div class="fb-like" data-href="http://onlinearizonahomes.info" data-send="true" data-width="450" data-show-faces="true" data-font="arial"></div></div> i thinking of calling template part within post <?php get_template_part( 'social' ); ?> cannot use php within post without plugin (i'd avoid using plugin this). an alternative using javascript create html in post, seems unnecessary use of javascript. what best way add html within wordpress post? you can use shortcode. add functions.php function cudjex_fbshare( $atts, $content = null ) { global $post; $link = get_permalink($post->id); return '<div id="socialwrap...
Read more

Firefox SVG shape not printing when it has stroke -

Image
i having issue svg shapes have stroke , trying them print in firefox. this simplest example: <svg xmlns="http://www.w3.org/2000/svg" version="1.1" xmlns:xlink="http://www.w3.org/1999/xlink"> <rect x="0" y="0" rx="15" ry="15" width="300" height="400" style="stroke:black;stroke-width:5;" fill="black" /> <circle id="firstcircle" cx="50" cy="50" r="30" stroke="black" stroke-width="2" fill="white" style="opacity:0.75;"/> <circle id="secondcircle" cx="50" cy="150" r="30" fill="white" style="opacity:0.75;"/> </svg> when try , print first shape 1 of 2 things: it not show @ all it shows off center in bounding box. the second shape no stroke shows expected, expected. when displaying on scre...
Read more

How to mention the localhost in android -

i trying call localhost in android application. unfortunately not possible used several methods emulator local host http://10.0.2.2/android/dbconnection.php and network ip address http://192.168.1.xx/android/dbconnection.php also http://localhost:8080/android/dbconnection.php but nothing connected localhost. purpose refer stackoverflow answers not working i facing same problem when worked on localhost. referred stackoverflow answer got solution create new emulator works fine in new emulator.
Read more