tensorflow optimizing sparse_tensor_dense_matmul operation on GPU -


is optimizing sparse_tensor_dense_matmul operation possible in tensorflow on gpu? use tensoflow 1.2.1 cuda 8. error example:

import tensorflow tf  tf.device('/gpu:0'):     st = tf.sparsetensor(         tf.constant([[0, 0], [1, 1]], dtype=tf.int64),         tf.constant([1.2, 3.4], dtype=tf.float32),         tf.constant([2, 2], dtype=tf.int64)     )      v = tf.variable([[1.0, 0.0], [0.0, 1.0]], dtype=tf.float32)     st = tf.sparse_tensor_dense_matmul(st, v)     st = tf.reduce_min(st)     optimizer = tf.train.adamoptimizer()     trainer = optimizer.minimize(st)  tf.session() sess:     print(sess.run(trainer))

results in following error:

traceback (most recent call last):   file "test_tf3.py", line 18, in <module>     print(sess.run(trainer))   file "/media/awork/home/astepochkin/drecs/repo/env/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 895, in run     run_metadata_ptr)   file "/media/awork/home/astepochkin/drecs/repo/env/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1124, in _run     feed_dict_tensor, options, run_metadata)   file "/media/awork/home/astepochkin/drecs/repo/env/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1321, in _do_run     options, run_metadata)   file "/media/awork/home/astepochkin/drecs/repo/env/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1340, in _do_call     raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.invalidargumenterror: cannot assign device operation 'gradients/sparsetensordensematmul/sparsetensordensematmul_grad/strided_slice_1': not satisfy explicit device specification '/device:gpu:0' because no supported kernel gpu devices available.      [[node: gradients/sparsetensordensematmul/sparsetensordensematmul_grad/strided_slice_1 = stridedslice[index=dt_int32, t=dt_int64, begin_mask=1, ellipsis_mask=0, end_mask=1, new_axis_mask=0, shrink_axis_mask=2, _device="/device:gpu:0"](const, gradients/sparsetensordensematmul/sparsetensordensematmul_grad/strided_slice_1/stack, gradients/sparsetensordensematmul/sparsetensordensematmul_grad/strided_slice_1/stack_1, gradients/sparsetensordensematmul/sparsetensordensematmul_grad/strided_slice_1/stack_2)]]

it may make sense disable hard device placement:

import tensorflow tf  tf.device('/gpu:0'):     st = tf.sparsetensor(         tf.constant([[0, 0], [1, 1]], dtype=tf.int64),         tf.constant([1.2, 3.4], dtype=tf.float32),         tf.constant([2, 2], dtype=tf.int64)     )      v = tf.variable([[1.0, 0.0], [0.0, 1.0]], dtype=tf.float32)     st = tf.sparse_tensor_dense_matmul(st, v)     st = tf.reduce_min(st)     optimizer = tf.train.adamoptimizer()     trainer = optimizer.minimize(st)  tf.session(config=tf.configproto(allow_soft_placement=true)) sess:     sess.run(tf.global_variables_initializer())     print(sess.run(trainer))

you can log device placements, may useful figuring out whether kernels care on gpu.

there host memory fake gpu kernels registered int32 strided slice, not int64. open pull request / feature request on github add int64 host memory kernels (effectively copying int32 versions) if need/want hard device placement.

for background, strided slice getting used in gradient of sparsetensordensematmul. there's no benefit running these kinds of indexing operations on gpu, registered gpu kernels run on cpu in order avoid kinds of hard device placement bookkeeping issues you've run into.


Comments

Post a Comment

Popular posts from this blog

python - Operations inside variables -

this question has answer here: evaluating mathematical expression in string 12 answers i want make python script finds maximum possible output using 2 operations. problem says there syntax error @ 2nd last line, in b in there. how can fix it? x = 0 = int(input("1st number:")) c = int(input("2nd number:" )) e = int(input("3rd number:" )) in range(4): if == 0: b = "+" elif == 1: b = "-" elif == 2: b = "/" else: b = "*" j in range(4): if j == 0: d = "+" elif j == 1: d = "-" elif j == 2: d = "/" else: d = "*" k = b c d e print(k) if you're after variables e on single line, couldn...
Read more

Generic Map Parameter java -

i having lot of methods of type: public static list<eduusers> getdetails(class c, map<string, integer> params) { list<eduusers> details = null; session session = hibernateutil.getsessionfactory().opensession(); transaction tx = null; try { tx = session.begintransaction(); criteria cr = session.createcriteria(c); (map.entry<string, integer> entry : params.entryset()) { cr.add(restrictions.eq(entry.getkey(), entry.getvalue())); } details = cr.list(); tx.commit(); } catch (exception asd) { log.debug(asd.getmessage()); if (tx != null) { tx.commit(); } } { session.close(); } return details; } i trying have generic method them , have written far: public static <t> list<t> getdetails(class<t> c, class<t> b) { l...
Read more

arrays - What causes a java.lang.ArrayIndexOutOfBoundsException and how do I prevent it? -

what arrayindexoutofboundsexception mean , how rid of it? here code sample triggers exception: string[] name = {"tom", "dick", "harry"}; for(int = 0; i<=name.length; i++) { system.out.print(name[i] +'\n'); } your first port of call should documentation explains reasonably clearly: thrown indicate array has been accessed illegal index. index either negative or greater or equal size of array. so example: int[] array = new int[5]; int boom = array[10]; // throws exception as how avoid it... um, don't that. careful array indexes. one problem people run thinking arrays 1-indexed, e.g. int[] array = new int[5]; // ... populate array here ... (int index = 1; index <= array.length; index++) { system.out.println(array[index]); } that miss out first element (index 0) , throw exception when index 5. valid indexes here 0-4 inclusive. correct, idiomatic for statement here be: for (int index = 0; index <...
Read more