lda Y_Table_Hi,y sta $fc lda Y_Table_lo,y sta $fb ldy X_Table,x lda BitMask,x ora ($fb),y sta ($fb),y
In terms of gameplay, do you think you're going to avoid all that bullets during the gameplay? I would try to remove half of the bullets, and try to play with the speed.
#1: How is the plotter set up, not using a shitload of jsr and stuff I hope. Also as it's <256 pixels, you should get away with a: lda Y_Table_Hi,y sta $fc lda Y_Table_lo,y sta $fb ldy X_Table,x lda BitMask,x ora ($fb),y sta ($fb),y
Second, the bullets seem to follow the path of an expanding circle. Could this be handled by a adapted Bresenham's circle algorithm (only drawing in two lower quadrants and limiting the number of pixels to match what you have on your youtube video)?
plott: lda Lobyte sta $fb sta wiperoutine1+1 lda Hibyte sta $fc sta wiperoutine1+2 //plottit rts wiperoutine: lda #$00 wiperoutine1: sta $2000 wiperoutine2: sta $2000 //.... rts