There is dialog configuration file named tag.config. You can find it in CuteSoft_Client/Cuteeditor/Dialogs folder.

 
The following code specifies which file will be used when the user double click the images in the editor.

<add pattern="IMG" tab="IMG" text="[[Image]]" control="Tag_Image.ascx" />

You can change its value and and point it to your own custom dialog.

Hope it helps.

Let me know if you have any further questions.