Kumar, who is on a week-long trip to Pakistan, travelled along with his entourage that included Rajya Sabha MP N K Singh, to Taxila, the UNESCO World heritage site that is a part of the shared history of India and Pakistan.
As he visited the site of historic glory, 32 kilometres northwest to Islamabad, he drew in it parallels to Nalanda, the great centre of ancient learning in his own state Bihar.
"A visit to Taxila is always overpowering. It is a journey three thousand years old in time, to a place which was a great centre of learning and intellectual pursuits with somewhat parallel of its kind situated at Nalanda in the state of Bihar," Kumar wrote in the visitors' book at Taxila.
The city that dates back to the Gandhara period, also houses the ruins of the Gandhara city which was regarded as an important Hindu and Buddhist centre.
"Nalanda and Taxila may be separated by geography but they are linked by history as ancient centres of knowledge and it is this bond of knowledge we shall aspire to revive. Anyone coming from Bihar a visit to Taxila will always be emotive and overpowering. I can't be an exception," he wrote.
From the historic site, Kumar then drove to the revered Katasraj temples in Punjab province and witnessed first hand the restoration of its centuries old holy pond, that has recently been completed.
He expressed hope that the restoration of the ancient shrine will go a long way in strengthening cross-cultural relations between the two countries.
The temples at the Katasraj site were built by Hindu kings almost 1000 years ago.
The temples and the ancient pond the site houses was restored on the Pakistan government's intervention recently after media reports highlighted that the sacred pond was drying up because of excessive use of groundwater in the region by a cement factory.
"More than a thousand year old Katasraj temples and its restoration in recent years is an excellent example of our cultural links and the deep desire to restore that spirit of cultural oneness," he wrote at the visitors' book at the site.
On Tuesday night, President Asif Ali Zardari hosted a special Diwali dinner for Kumar and his delegation and pitched for "friendly, cooperative and good neighbourly relations" with India.
Welcoming Kumar and his delegation, Zardari extended his greetings for Diwali and said such parliamentary exchanges between the two sides are highly encouraging and will contribute to the deepening of people-to-people contacts.
Speaking at the Institute of Strategic Studies, Kumar also shared his experiences of governing Bihar and achieving a much talked about turnaround in the condition of the state.
Kumar listed the steps he had taken to control law and order and provide governance at the grassroots level as the two main factors for the turnaround in Bihar. He said he had made it clear that "the guilty would not be spared".
"As part of our efforts to control corruption, we even seized huge bungalows that were built by a former bureaucrat and a former senior police official," Kumar, who set aside a prepared text in English and addressed the gathering in Hindi, said, adding that his government passed the Right to Public Services Act as part of measures to ensure good governance.
"This has ensured the time-bound delivery of services like the issuing of certificates. Any delay in providing services results in disciplinary action against officials," he said.
He also said that courts were set up to conduct speedy trials and the police force was overhauled, and this had led to the conviction of over 74,000 criminals.Image: Nitish Kumar visit's Mohenjodaro in the province of Sindh, Pakistan