Marc McNulty gave the home side the advantage in a hard-fought match after just three minutes with a shot from the edge of the box.
Though it was not long before Jamille Matt found the net with a header in the ninth minute to equalise for Plymouth.
Mat scored his second of the night - an overhead kick - to give the visitors the edge just ten minutes later.
AFC Wimbledon beat Plymouth to secure League One promotion
Portsmouth's Marc McNulty is fouled by Plymouth's Peter Hartley resulting in a penalty
Image credit: Reuters
But Argyle gave their lead away after McNulty was tripped in the area by Peter Hartley, and awarded a penalty which he converted in the 51st minute.
The teams go forward to their next meeting on 15 May level, though with a slight advantage to Portsmouth, to decide who will go through to the play-off final at Wembley two weeks later.
Oxford and Bristol Rovers grab promotion as Accrington Stanley blow it
Hughes winner keeps Accrington on course for automatic promotion