Thousands of devotees today took a holy dip in the Ganga and other rivers on the occasion of Mahashivratri at various pilgrim destinations.