cat file1.txt | anew file2.txt

this will take the unique subdomains from file1 which are not in file2 and save it to file2 removing all other duplucates