I believe retroarch (the emulator running n64 games) saves config on exit, and when you exit the game via Polycade AGS, it kills the emulator process instead of the emulator getting to do it's "safe shutdown" procedure.
So, you'll want to do your controller mapping and then either find the save function in retroarch's menus (main menu > configuration file > save current configuration), or just exit retroarch via the retroarch menu.
Maybe more info here:
https://forums.libretro.com/t/retroarch-settings-not-saving/30652