Go channel infinite loop -
i trying catch errors group of goroutines using channel, channel enters infinite loop, starts consuming cpu.
func unzipfile(f *bytes.buffer, location string) error { zipreader, err := zip.newreader(bytes.newreader(f.bytes()), int64(f.len())) if err != nil { return err } if err := os.mkdirall(location, os.modeperm); err != nil { return err } errorchannel := make(chan error) errorlist := []error{} go errorchannelwatch(errorchannel, errorlist) filewaitgroup := &sync.waitgroup{} _, file := range zipreader.file { filewaitgroup.add(1) go writezipfiletolocal(file, location, errorchannel, filewaitgroup) } filewaitgroup.wait() close(errorchannel) log.println(errorlist) return nil } func errorchannelwatch(ch chan error, list []error) { { select { case err := <- ch: list = append(list, err) } } } func writezipfiletolocal(file *zip.file, location string, ch chan error, wg *sync.waitgroup) { defer wg.done() zipfilehandle, err := file.open() if err != nil { ch <- err return } defer zipfilehandle.close() if file.fileinfo().isdir() { if err := os.mkdirall(filepath.join(location, file.name), os.modeperm); err != nil { ch <- err } return } localfilehandle, err := os.openfile(filepath.join(location, file.name), os.o_wronly|os.o_create|os.o_trunc, file.mode()) if err != nil { ch <- err return } defer localfilehandle.close() if _, err := io.copy(localfilehandle, zipfilehandle); err != nil { ch <- err return } ch <- fmt.errorf("test error") }
so looping slice of files , writing them disk, when there error report errorchannel
save error slice.
i use sync.waitgroup
wait goroutines , when done want print errorlist
, check if there error during execution.
the list empty, if add ch <- fmt.errorf("test")
@ end of writezipfiletolocal
, channel hangs up.
i not sure missing here.
1. first point, infinite loop:
citing golang language spec:
a receive operation on closed channel can proceed immediately, yielding element type's 0 value after sent values have been received.
so in function
func errorchannelwatch(ch chan error, list []error) { { select { case err := <- ch: list = append(list, err) } } }
after ch gets closed turns infinite loop adding nil
values list
.
try instead:
func errorchannelwatch(ch chan error, list []error) { err := range ch { list = append(list, err) } }
2. second point, why don't see in error list:
the problem call:
errorchannel := make(chan error) errorlist := []error{} go errorchannelwatch(errorchannel, errorlist)
here hand errorchannelwatch
errorlist
value. slice errorlist
not changed function. changed, underlying array, long append
calls don't need allocate new one.
to remedy situation, either hand slice pointer errorchannelwatch
or rewrite call closure, capturing errorlist
.
for first proposed solution, change errorchannelwatch
to
func errorchannelwatch(ch chan error, list *[]error) { err := range ch { *list = append(*list, err) } }
and call to
errorchannel := make(chan error) errorlist := []error{} go errorchannelwatch(errorchannel, &errorlist)
for second proposed solution, change call to
errorchannel := make(chan error) errorlist := []error{} go func() { err := range errorchannel { errorlist = append(errorlist, err) } } ()
3. minor remark:
one think, there synchronisation problem here:
filewaitgroup.wait() close(errorchannel) log.println(errorlist)
how can sure, errorlist isn't modified, after call close? 1 reason, can't know, how many values goroutine errorchannelwatch
still has process.
your synchronisation seems correct me, wg.done()
after send error channel , error values sent, when filewaitgroup.wait()
returns.
but can change, if later adds buffering error channel or alters code.
so advise @ least explain synchronisation in comment.
Comments
Post a Comment