What is difference between .hpi and .jpi of Jenkins plugins?

Both are supposed to be identical to that extend that Jenkins is renaming hpi to jpi when you install it manually as you said.

The reason why you see both in your JENKINS_HOME is the order in which plugins are loaded when Jenkins boots up: plugin.jpi gets precedence over plugin.hpi in case both are present. This is the way the upload installation makes sure the uploaded version will override the existing one after the restart.

Well I think its because Jenkins forked from Hudson so that is the 'H' in hpi. The J is obviously a change to that.

In terms of technology, the jpi plugins are generated using the gradle plugin architecture and hpi plugins are generated with the Maven architecture

Ultimately, as you have found both produce plugins which Jenkins can use. The vast majority are built using Maven but I am a fan of Gradle as it links nicely in with developing plugins in Groovy. You can also build in Groovy using Maven

Plugins as present in $JENKINS_HOME/plugins/ should always be using the .jpi suffix (with the basename being the plugin identifier). Normally Jenkins will enforce this naming pattern even when uploading a plugin manually, regardless of what filename you used for the upload, so I am not sure how you came to have a *.hpi file here, unless you directly copied it into this filesystem location.