If the Papacy abides in the spirit or intention of God/truth instead of the letter then it is infallible.
A parent can have a good intention towards raising their child regardless of the material and immaterial wealth that it possesses to help raise the child:
The parent can possess either:
A few ounces of locally grown and processed porridge oats and goats milk, and a wooden toy;
Or a multivitamin cereal meal with milk that is fortified with vitamins and minerals (as determined by biologists and nutritionists), and an electronic learning aid (developed by highly educated and knowledgeable people).
Whichever of these materials that the parent possess is largely irrelevant, so long as the intention of the parent toward the child is to do good by it, to raise it and help the child grow as best as they can. It's the attitude that matters more than the resources; it's the intention that matters not the details. The same applies to the church and to the papacy: so long as they adhere to the spirit of God, God's will, God's intention, God's love, then they are infallible; when they decide to adhere to the spirit of the law instead (e.g. eat fish on Fridays, go to prayer on Sundays, go to confession once a week) then they are no longer infallible. God first, facts second.