Both of the last two lines "break" the custom button with a command not implemented.
By changing -- var editdoc = editor1.GetDocument();
to -- var editdoc = editor.GetDocument();
I get by the first problem.
I'm not a java developer so really don't understand the "new" call. It appears that you have changed the attrubutes of the showModal call yet the old call displays the dialog window, but the "new" call breaks the java function.
Phil