As with all things, alt and enter switches between windowed mode and fullscreen
If you want a bigger window choose a different graphics filter like 3x. Most ppl like advmame2x best though.
There is mouse support, most likely you have a joystick/pad plugged in and thats taking priority or something. Certainly on my system my sidewinder freestyle pro is set to its weird tilt/analogue thing by default and that shoves the cursor on a corner of the window until I turn it onto normal dpad mode.
Pressing the arrow keys on the keyboard usually overrides the joypad and returns control to keyboard/mouse.