كتاب الأيمان
27  
The Book of Oaths
(6)
Chapter: The prohibition of persisting in an oath that will harm the family of the one who swears it, so long as not persisting in it does not involve anything unlawful
(6)
باب النَّهْىِ عَنِ الإِصْرَارِ، عَلَى الْيَمِينِ فِيمَا يَتَأَذَّى بِهِ أَهْلُ الْحَالِفِ مِمَّا لَيْسَ بِحَرَامٍ
Sahih Muslim 1655

Hammam b. Munabbih reported:

This is what Abu Huraira reported to us from Allah's Messenger (ﷺ), and he narrated a hadith and (one) of them is that Allah's Messenger (ﷺ) said: I swear by Allah, it is more sinful in Allah's sight for one of you to persist in an oath regarding his family than payment of its expiation which Allah has imposed upon him (for breaking the oath).
حَدَّثَنَا مُحَمَّدُ بْنُ رَافِعٍ، حَدَّثَنَا عَبْدُ الرَّزَّاقِ، حَدَّثَنَا مَعْمَرٌ، عَنْ هَمَّامِ بْنِ مُنَبِّهٍ، قَالَ هَذَا مَا حَدَّثَنَا أَبُو هُرَيْرَةَ، عَنْ رَسُولِ اللَّهِ صلى الله عليه وسلم فَذَكَرَ أَحَادِيثَ مِنْهَا وَقَالَ رَسُولُ اللَّهِ صلى الله عليه وسلم ‏ "‏ وَاللَّهِ لأَنْ يَلَجَّ أَحَدُكُمْ بِيَمِينِهِ فِي أَهْلِهِ آثَمُ لَهُ عِنْدَ اللَّهِ مِنْ أَنْ يُعْطِيَ كَفَّارَتَهُ الَّتِي فَرَضَ اللَّهُ ‏"‏ ‏.‏
Reference : Sahih Muslim 1655
In-book reference : Book 27, Hadith 38
USC-MSA web (English) reference : Book 15, Hadith 4071
  (deprecated numbering scheme)