Possible partial, but ugly answer:
Have struggled with the import cyclic dependency problem for a year. For a while, was able to decouple enough so that there wasn't an import cycle. My application uses plugins heavily. At the same time, it uses encode/decode libraries (json and gob). For these, I have custom marshall and unmarshall methods, and equivalent for json.
For these to work, the full type name including the package name must be identical on data structures that are passed to the codecs. The creation of the codecs must be in a package. This package is called from both other packages as well as from plugins.
Everything works as long as the codec package doesn't need to call out to any package calling it, or use the methods or interfaces to the methods. In order to be able to use the types from the package in the plugins, the plugins have to be compiled with the package. Since I don't want to have to include the main program in the builds for the plugins, which would break the point of the plugins, only the codec package is included in both the plugins and the main program. Everything works up until I need to call from the codec package in to the main program, after the main program has called in to the codec package. This will cause an import cycle. To get rid of this, I can put the codec in the main program instead of its own package. But, because the specific datatypes being used in the marshalling/unmarshalling methods must be the same in the main program and the plugins, I would need to compile with the main program package for each of the plugins. Further, because I need to the main program to call out to the plugins I need the interface types for the plugins in the main program. Having never found a way to get this to work, I did think of a possible solution:
First, separate the codec in to a plugin, instead of just a package
Then, load it as the first plugin from the main program.
Create a registration function to exchange interfaces with underlying methods.
All encoders and decoders are created by calls in to this plugin.
The plugin calls back to the main program through the registered interface.
The main program and all the plugins use the same interface type package for this.
However, the datatypes for the actual encoded data are referenced in the main program
with a different name, but same underlying type than in the plugins, otherwise the same import cycle exists. to do this part requires doing an unsafe cast. Wrote
a little function that does a forced cast so that the syntax is clean:
(<cast pointer type*>Cast(<pointer to structure, or interface to pointer to structure>).
The only other issue for the codecs is to make sure that when the data is sent to the encoder, it is cast so that the marshall/unmarshall methods recognize the datatype names. To make that easier, can import both the main program types from one package, and the plugin types from another package since they don't reference each other.
Very complex workaround, but don't see how else to make this work.
Have not tried this yet. May still end up with an import cycle when everything is done.
[more on this]
To avoid the import cycle problem, I use an unsafe type approach using pointers. First, here is a package with a little function Cast() to do the unsafe typecasting, to make the code easier to read:
package ForcedCast
import (
"unsafe"
"reflect"
)
// cast function to do casts with to hide the ugly syntax
// used as the following:
// <var> = (cast type)(cast(input var))
func Cast(i interface{})(unsafe.Pointer) {
return (unsafe.Pointer(reflect.ValueOf(i).Pointer()))
}
Next I use the "interface{}" as the equivalent of a void pointer:
package firstpackage
type realstruct struct {
...
}
var Data realstruct
// setup a function to call in to a loaded plugin
var calledfuncptr func(interface)
func callingfunc() {
pluginpath := path.Join(<pathname>, "calledfuncplugin")
plug, err := plugin.Open(pluginpath)
rFunc, err := plug.Lookup("calledfunc")
calledfuncptr = rFunc.(interface{})
calledfuncptr (&Data)
}
//in a plugin
//plugins don't use packages for the main code, are build with -buildmode=plugin
package main
// identical definition of structure
type realstruct struct {
...
}
var localdataptr *realstruct
func calledfunc(needcast interface{}) {
localdataptr = (*realstruct)(Cast(needcast))
}
For cross type dependencies to any other packages, use the "interface{}" as a void pointer and cast appropriately as needed.
This only works if the underlying type that is pointed to by the interface{} is identical wherever it is cast. To make this easier, I put the types in a separate file. In the calling package, they start with the package name. I then make a copy of the type file, change the package to "package main", and put it in the plugin directory so that the types are built, but not the package name.
There is probably a way to do this for the actual data values, not just pointers, but I haven't gotten that to work right.
One of the things I have done is to cast to an interface instead of a datatype pointer. This allows you to send interfaces to packages using the plugin approach, where there is an import cycle. The interface has a pointer to the datatype, and then you can use it for calling the methods on the datatype from the caller from the package that called in to the plugin.
The reason why this works is that the datatypes are not visible outside of the plugin. That is, if I load to plugins, which are both package main, and the types are defined in the package main for both, but are different types with the same names, the types do not conflict.
However, if I put a common package in to both plugins, that package must be identical and have the exact full pathname for where it was compiled from. To accommodate this, I use a docker container to do my builds so that I can force the pathnames to always be correct for any common containers across my plugins.
I did say this was ugly, but it does work. If there is an import cycle because a type in one package uses a type in another package that then tries to use a type from the first package, the approach is to do a plugin that erases both types with interface{}. You can then make method and function calls back and forth doing the casting on the receiving side as needed.
In summary:
Use interface{} to make void pointers (that is, untyped).
Use the Cast() to force them to a pointer type that matches the underlying pointer. Use the plugin type localization so that types in the package main in separate plugins, and in the main program do not conflict If you use a common package between plugins, the path must be identical for all built plugins and the main program. Use the plug package to load the plugins, and exchange function pointers
For one of my issues I'm actually calling from a package in the main program out to a plugin, just to be able to call back to another package in the main program, avoiding the import cycle between the two packages. I ran in to this problem using the json and gob packages with custom marshaller methods. I use the types that are custom marshalled both in my main program, and in other plugins, while at the same time, I want the plugins to be built independent of the main program. I accomplish this by using a package for json and gob encode/decode custom methods that is included both in the main program and the plugins. However, I needed to be able to call back to the main program from the encoder methods, which gave me the import cycle type conflict. The above solution with another plugin specifically to solve the import cycle works. It does create an extra function call, but I have yet to see any other solution to this.
Hope this helps with this issue.
