How can I limit the bandwidth usage of over-IP KVM ?

You can configure the Network Transfer Rate in Device Management > Network for using limited bandwidth. 

But FPS will become less that might lead to the lag of video playing during remote access.