Short answer
In everyday speech, simply non-Muslims, or by their faith: Christians, Jews, Hindus. The Quran uses kafir (one who rejects faith in Islam) and mushrik (one who worships others besides God) as belief terms, and calls Jews and Christians Ahl al-Kitab, People of the Book. Using kafir as a slur to hurt someone is sinful.
Kafir describes a belief, not a curse word. Hanafi texts state that a Muslim who calls a Jew or a Zoroastrian (Majusi) O kafir in a way that hurts him is sinful, and the same reasoning covers any peaceful non-Muslim, since hurting them is forbidden. Calling a Muslim kafir is far graver.
The four schools agree on this.
Explanation
Kafir comes from a root meaning to cover. Arabic even calls a farmer a kafir because he covers seed with soil, and the Quran uses the word that way once. In belief it means someone who does not accept Islam's message, covering over the truth in Islam's view. Every religion has a word for those outside it, and kafir is Islam's.
Mushrik means one who commits shirk: worshipping others alongside God. The Quran mentions the People of the Scripture and the polytheists as two groups, which is why Jews and Christians have their own rulings in fiqh. See who are the People of the Book and do Muslims believe Christians are polytheists.
Fiqh has other words based on a person's legal status: dhimmi for a non-Muslim citizen of a historical Muslim state, and mu'ahad for someone living under a covenant of peace. Today a Muslim's non-Muslim neighbours and colleagues are covered by the agreements of citizenship and residence, and the Prophet ﷺ warned severely against harming such people.
Words also carry manners. The Quran forbids insulting nicknames, and the Prophet ﷺ said he was not sent to curse but as a mercy. So in normal life Muslims say non-Muslim, or Christian, Jewish, Hindu and so on. They keep theological terms for theology, and never throw them at people.
The evidence
Quran, Surah Al-Bayyina 98:1
بِسْمِ ٱللَّهِ ٱلرَّحْمَٰنِ ٱلرَّحِيمِ لَمْ يَكُنِ ٱلَّذِينَ كَفَرُوا۟ مِنْ أَهْلِ ٱلْكِتَٰبِ وَٱلْمُشْرِكِينَ مُنفَكِّينَ حَتَّىٰ تَأْتِيَهُمُ ٱلْبَيِّنَةُ ﴿١﴾
Those who disbelieved among the People of the Scripture and the polytheists were not to be parted [from misbelief] until there came to them clear evidence -
The Quran treats the People of the Book and polytheists as distinct groups.
Quran, Surah Al-Hujuraat 49:11
يَٰٓأَيُّهَا ٱلَّذِينَ ءَامَنُوا۟ لَا يَسْخَرْ قَوْمٌۭ مِّن قَوْمٍ عَسَىٰٓ أَن يَكُونُوا۟ خَيْرًۭا مِّنْهُمْ وَلَا نِسَآءٌۭ مِّن نِّسَآءٍ عَسَىٰٓ أَن يَكُنَّ خَيْرًۭا مِّنْهُنَّ ۖ وَلَا تَلْمِزُوٓا۟ أَنفُسَكُمْ وَلَا تَنَابَزُوا۟ بِٱلْأَلْقَٰبِ ۖ بِئْسَ ٱلِٱسْمُ ٱلْفُسُوقُ بَعْدَ ٱلْإِيمَٰنِ ۚ وَمَن لَّمْ يَتُبْ فَأُو۟لَٰٓئِكَ هُمُ ٱلظَّٰلِمُونَ ﴿١١﴾
O you who have believed, let not a people ridicule [another] people; perhaps they may be better than them; nor let women ridicule [other] women; perhaps they may be better than them. And do not insult one another and do not call each other by [offensive] nicknames. Wretched is the name of disobedience after [one's] faith. And whoever does not repent - then it is those who are the wrongdoers.
Using labels to insult people is forbidden.
Sahih Muslim 2599
Sahih... إني لم أبعث لعانا وإنما بعثت رحمة ...
"I was not sent to curse; I was sent only as a mercy."
The Prophet ﷺ refused to curse even those who opposed him.
Sahih al-Bukhari 6103
Sahih... إذا قال الرجل لأخيه يا كافر فقد باء به أحدهما ...
"If a man says to his brother, O kafir, then it returns upon one of them."
Calling a Muslim kafir is a grave matter.
Where the ruling is recorded
- al-Durr al-Mukhtar, Kitab al-Hudud, Bab al-Ta'zir. Citing al-Qunya: whoever says to a Jew or Majusi O kafir is sinful if it hurts him.
Checked by our team. For a more accurate answer to your own situation, consult a scholar. Quran translation by Sahih International. Hadith Arabic from the standard editions of the six books and the Muwatta, with gradings as recorded there; the English is our own rendering. Spotted a mistake? Tell us.