linux - How to filter out the duplicate id between 02 files -
i have 02 text files bellow format:
file 1:
2017-08-16 00:00:00,115 - [info] transactions: 123456788 id: 123456 2017-08-16 00:00:00,115 - [info] transactions: 123456789 id: 123457
file 2:
123456 123457 123458 123459
the goal: records file1 without id in file2
the commands line , result tried:
- 1st command line:
grep -vf file2 file1
- 2nd command line:
comm -23 <(sort file1) <(sort file2)
the both of command worked there 3 millions records in file1 , 1 millions records in file2. 1st command can complete if there not records can not complete 3 millions. 2nd command faster 1st , can completed when executed manually in ssh console did not work bash script. error has showed "syntax error @ "("
any idea solve , complete goal ?
awk 'nr==fnr{a[$1];next} !($nf in a)' file2 file1
Comments
Post a Comment