Obama, pilloried on the 2008 campaign trail for offering to sit down with US enemies, will also remember the backlash sparked when he shook hands with late Venezuelan president Hugo Chavez.
However, some Iran watchers say that even a quick meeting between Obama and Rouhani could offer the stop-start talks between the permanent five members of the UN Security Council plus Germany and Iran, a jolt of momentum before their expected resumption within weeks.
Washington and Iran also have much to discuss on Syria — where they are backing rival sides in the brutal civil war.
As they weigh the risks of a meeting, Obama and Rouhani must also consider the flipside — that a chance not taken may not come again.
In 2000, US president Bill Clinton went to the UN hoping to meet then-Iranian president Mohammad Khatami at another elusive moment of promise in the Washington-Tehran chill.
The encounter that had been expected never happened — leaving bruised feelings on both sides.
Within months, hopes of a breakthrough were crushed with hardliners again on the rise in Tehran.