Why does the cool feeling of silk usually fade after your skin stays on the same spot for a while?

Answer: Fabric and skin equalize

Silk uses up coolness — Silk is not storing a finite supply of coldness that gets used up. The first moment feels cool because heat is leaving skin rapidly. Once the contact patch has warmed, the thermal gradient shrinks, so there is less new heat-flow signal for your cold receptors to report.

Fabric and skin equalize ✓ — Correct. The silk starts by accepting heat from warmer skin, so the contact patch feels cool. After seconds or minutes, the surface near your skin warms toward skin temperature and the heat-flow rate drops. This is why a cool pillowcase is a first-contact effect, not a tiny air conditioner that keeps removing heat forever.

Surface texture changes — The silk surface does not need to become rougher or smoother for the cool feeling to fade. The same texture can feel cool at first and neutral later because the contact patch has warmed. The useful surprise is that the sensation can change even when the material and its feel under your fingers have not.