encoding - PHP writes garbled UTF8 characters to output file -


[please see below answer]

i using preg_match_all extract hashtags strings,for example:

#tree#ztdf #n4# night

contains hashtags: tree, ztdf, n4, night

strings can language characters, emojis. therefore, enabled utf-8 flag (/u) in preg_match_all:

preg_match_all('/#([\pl\p{mn}]+)/u', $media_caption,  $matches);

however, characters wrongly matched byte sequences:

enter image description here

i read problem preg_match_all, utf-8 encoding , php here. tried add additional utf-8 flag (*utf8) pcre:

preg_match_all('(*utf8)/#([\p{l}\p{mn}]+)/u', $media_caption,  $matches)

.. getting error

syntax error, unexpected 'enabled' t-flag

anyone knows how can extract #hashtags utf-8 character using preg_match_all?

[edit]

ok.. day, problem: realized yesterday, garbled characters got after json_decode() looking @ output windows command line, can't handle utf8. today run program using git bash console , - shows input preg_match_all looking fine in utf8. - after this, no problems: str_replace(array("\r\n", "\r", "\n",","), ";", $media_caption); (replace linebreaks) - , no problems after this: preg_replace('!\s+!u', ' ', $media_caption); (replace multiple space characters one) - funny part: looks fine after this: preg_match_all('/#([\p{l}\p{mn}]+)/u', $media_caption, $matches);

for example, var_dump following string in git bash:

 string(15) "presadebuendía"

.. in written csv/txt this: presadebuend㮡 while embalse de buendía correctly written file.

i looking parts of code may mess character encoding during data processing. far, have tried:

  • header('content-encoding: utf-8');
  • header('content-type: text/csv; charset=utf-8');
  • mb_internal_encoding("utf-8"); , replacing fopen function:
function utf8_fopen_read($filename) {      $fc = iconv('windows-1250', 'utf-8', file_get_contents($filename));      $handle=fopen("php://memory", "rw");      fwrite($handle, $fc);      fseek($handle, 0);      return $handle;  }

.. none of solved issue.

thank commenting. apologize pointing in wrong direction: preg_match_all , other regex functions not problem messing characters. couple of things confused me (such windows command line not being able output utf8). in end, there 1 issue in code:

  • before writing strings file, used strtolower function, reduces lowercase, including special characters such í (\u00e). solution use mb_strtolower instead, limited alphabetic characters.

of course, couldn't spot problem because didn't include specific code part in question! during searching problem, added

  • header('content-encoding: utf-8');
  • header('content-type: text/csv; charset=utf-8');
  • mb_internal_encoding("utf-8");

to php-script file, doesn't seem have effect on output file. anyway, solved problem. thank you!


Comments

Post a Comment

Popular posts from this blog

python - Operations inside variables -

this question has answer here: evaluating mathematical expression in string 12 answers i want make python script finds maximum possible output using 2 operations. problem says there syntax error @ 2nd last line, in b in there. how can fix it? x = 0 = int(input("1st number:")) c = int(input("2nd number:" )) e = int(input("3rd number:" )) in range(4): if == 0: b = "+" elif == 1: b = "-" elif == 2: b = "/" else: b = "*" j in range(4): if j == 0: d = "+" elif j == 1: d = "-" elif j == 2: d = "/" else: d = "*" k = b c d e print(k) if you're after variables e on single line, couldn...
Read more

Generic Map Parameter java -

i having lot of methods of type: public static list<eduusers> getdetails(class c, map<string, integer> params) { list<eduusers> details = null; session session = hibernateutil.getsessionfactory().opensession(); transaction tx = null; try { tx = session.begintransaction(); criteria cr = session.createcriteria(c); (map.entry<string, integer> entry : params.entryset()) { cr.add(restrictions.eq(entry.getkey(), entry.getvalue())); } details = cr.list(); tx.commit(); } catch (exception asd) { log.debug(asd.getmessage()); if (tx != null) { tx.commit(); } } { session.close(); } return details; } i trying have generic method them , have written far: public static <t> list<t> getdetails(class<t> c, class<t> b) { l...
Read more

arrays - What causes a java.lang.ArrayIndexOutOfBoundsException and how do I prevent it? -

what arrayindexoutofboundsexception mean , how rid of it? here code sample triggers exception: string[] name = {"tom", "dick", "harry"}; for(int = 0; i<=name.length; i++) { system.out.print(name[i] +'\n'); } your first port of call should documentation explains reasonably clearly: thrown indicate array has been accessed illegal index. index either negative or greater or equal size of array. so example: int[] array = new int[5]; int boom = array[10]; // throws exception as how avoid it... um, don't that. careful array indexes. one problem people run thinking arrays 1-indexed, e.g. int[] array = new int[5]; // ... populate array here ... (int index = 1; index <= array.length; index++) { system.out.println(array[index]); } that miss out first element (index 0) , throw exception when index 5. valid indexes here 0-4 inclusive. correct, idiomatic for statement here be: for (int index = 0; index <...
Read more