The reigning world champion clocked a 1m34.944s on the soft tyres to outpace Rosberg by 0.233s with Sebastian Vettel four tenths further back.
Drivers struggled with low grip levels early in FP2 on Sepang's brand new surface, with track temperatures soaring to 56C, but conditions improved as more rubber was put down.
Hamilton set the early pace on the hard tyres, clocking a 1m35.956s, six tenths quicker than Rosberg on the same rubber.
Malaysian Grand Prix
Podcast: Malaysian Grand Prix review
05/10/2016 AT 08:59
They then switched to the soft tyres for qualifying simulations with Hamilton edging out his team-mate again.
Attention then switched to gathering long run data for Sunday's race with the Mercedes duo running all three compounds between them.
Kimi Raikkonen reported there was "something not correct with the car" and retreated back to the pits so his Ferrari team could investigate.

Lewis Hamilton aims to bounce back at Malyasian GP

Image credit: Eurosport

The Finn rejoined and set the fourth quickest time on his second attempt on the softs, 0.2s adrift of team-mate Vettel.
Red Bull's Max Verstappen was fifth quickest, one second off the pace, ahead of Sergio Perez and Fernando Alonso.
Alonso continued to run Honda's updated power unit, after encountering a smooth session in FP1, but the plan is to revert to the previous spec for the remainder of the weekend.
Daniel Ricciardo, Nico Hulkenberg and Jenson Button completed the top 10.
Renault's Jolyon Palmer was 12th, 1.996s off the pace, while his team did a good job to repair team-mate Kevin Magnussen's car and get the Dane out midway through the session.
He did not set a timed lap in FP1 after his car was heavily damaged when it caught fire in the pitlane after his installation laps.
But he managed 19 laps in FP2, including a qualifying simulation on soft tyres, to end the session 19th.
FP2 TIMES
Malaysian Grand Prix
'Something just doesn't feel right' - Furious Hamilton seethes after engine failure
02/10/2016 AT 10:55
Malaysian Grand Prix
Ricciardo wins Malaysian GP after Hamilton suffers dramatic engine failure
02/10/2016 AT 08:57