Delgine 3D Tools & Content DeleD Community Edition
Forums
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Better Plugin-Interface?
Goto page Previous  1, 2, 3  Next
 
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies.    DeleD Community Edition Forum Index -> DeleD Community Edition
View previous topic :: View next topic  
Author Message
Jeroen
Site Admin


Joined: 07 Aug 2004
Posts: 5332
Location: The Netherlands

PostPosted: Thu Jan 06, 2011 4:33 pm    Post subject: Reply with quote

Ah, fact is that there aren't many plugin coders out there, due to the fact the plugin architecture doesn't suit most of the needs of coders out there. One of the main reasons for you to create a better one in the first place. Wink Also, not all coders are also technical designers. So... I think you won't get much response at first until there's actually something to work with and people start to see the value of the new architecture.

Perhaps it's a good idea to dive a little deeper into the API stuff. Can you tell us a little more about what you have in mind?
_________________
Check out Figuro, our online 3D app! More powerful 3D tools for free.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
chrisu
DeleD PRO user


Joined: 11 Sep 2009
Posts: 53
Location: Austria

PostPosted: Thu Jan 06, 2011 5:18 pm    Post subject: Reply with quote

Jeroen wrote:
Perhaps it's a good idea to dive a little deeper into the API stuff. Can you tell us a little more about what you have in mind?


Sure! (Btw., I updated the image of the concept.)

Firstly I noticed that there isn't anything behind (or within) DeleD that seems to be "isolated" as an API usually is. I mean, if there was one, then DeleD would use it itself and kinda "wrap" around it. But we definitely need some kind of API to have a defined usage of DeleD's internals from a plugin.
As we don't want reinvent DeleD from A to Z it's just an implication that this API in our situation will follow the opposite way and wrap around "DeleD". In other words the API will be just be a kind of a plugin which keeps up communication with DeleD but trying to not hinder it. So my idea is, that the API will further act as some kind of job-queue. Plugins - the users of the API - will do API-requests which API queues and DeleD periodicly is checking the queue if anything needs to be done raising the API's priority if that is so and reducing it if the queue gets empty again.

One thing I have to know for that is if DeleD by design is multithreaded or not, because that's something which has a major impact on if we need a queue model or can go without one.

Edit: I just checked the code for "CreateThread" and found nothing - I guess DeleD is single-threaded, which means that we wouldn't have to primarily create a queue-solution for the API. Maybe just for "Cooperator"-plugins but I have to think about this.

When a plugin is initially loaded it registers itself in DeleD and in the API and will so be part of the whole application-complex.

Edit: Maybe a short explanation why we need an API. Firstly there's a need for performance and tranferring those huge XML-trees doesn't really scale according to performance - if you know what I mean. So any plugin should only do things and transfer/use data that it really needs to. Second thing is that anyone uses her favorite programming language and there are so much things that are different between those all that the interface needs to be generic as can be. A typical API-Dll as it's usually done should do this job nicely.

Hm, I hope that all is understandable, for youse. It's always kinda hard to explain things grown in mind to others Smile

Cheers,
Chrisu.
Back to top
View user's profile Send private message Visit poster's website
AWM Mars
Member


Joined: 06 Jan 2010
Posts: 1195
Location: Wilts England

PostPosted: Thu Jan 06, 2011 7:32 pm    Post subject: Reply with quote

chrisu wrote:

Hm, I hope that all is understandable, for youse. It's always kinda hard to explain things grown in mind to others Smile

Cheers,
Chrisu.

I suspect Da Vinci must have felt the same way Smile
_________________
Politeness is priceless when received, cost nothing to own or give, yet some cannot afford.

Checkout:
http://www.awm.mars.yourinside.com/
http://www.bccservices.co.uk
http://www.localtradecheck.co.uk
Back to top
View user's profile Send private message Visit poster's website
Jeroen
Site Admin


Joined: 07 Aug 2004
Posts: 5332
Location: The Netherlands

PostPosted: Sat Jan 08, 2011 8:37 am    Post subject: Reply with quote

Just a few random remarks & questions:

chrisu wrote:
But we definitely need some kind of API to have a defined usage of DeleD's internals from a plugin.


I imagine the API being the technical part that communicates with DeleD's internal structure (as in, the lists of primitives, the selections etc) and provides an easy way for developers to make changes to that internal structure. Plugins talk to the API. The API talks to DeleD. That's what you're saying, too, right?

Would that API be a DLL?

chrisu wrote:
In other words the API will be just be a kind of a plugin which keeps up communication with DeleD but trying to not hinder it.


Indeed, it's very important not to interfere with the existing tools. They should not have to be modified in any way, because that would mean a major refactoring.

chrisu wrote:

So my idea is, that the API will further act as some kind of job-queue. Plugins - the users of the API - will do API-requests which API queues and DeleD periodicly is checking the queue if anything needs to be done raising the API's priority if that is so and reducing it if the queue gets empty again.


Job-queue's and the need to check things periodically may make things overly complicated and can introduce unexpected behavior (timing issues etc), at least in my experience, so if we can avoid that, please do. When editing a 3d scene, users usually perform one task at a time. Is there any reason why things couldn't be handled real-time?

chrisu wrote:

One thing I have to know for that is if DeleD by design is multithreaded or not, because that's something which has a major impact on if we need a queue model or can go without one.


There are places where DeleD uses threads, the raytracer being one of them, IIRC. But overall, nope, threads are avoided.

chrisu wrote:

When a plugin is initially loaded it registers itself in DeleD and in the API and will so be part of the whole application-complex.


There will probably be different kinds of plugins. Will they all have their own specific place in the DeleD UI? For example, importer/exporter plugins can be handled just the way they are now, being in a menu of their own, with their own UI handling things. I would adopt that because users are already used to that.

Tool plugins are another matter though. It would be nice if a plugin could nest itself within the existing Geometry Toolbox. Any thoughts on that?

chrisu wrote:

Edit: Maybe a short explanation why we need an API. Firstly there's a need for performance and tranferring those huge XML-trees doesn't really scale according to performance - if you know what I mean.


Yes, I do. The reason I choose to pass the XML trees, is because I intended plugins to be used for import/export uses only, in which case passing the entire structure is quite alright. But of course, if we want to create our own custom geometry tools and do other specific things, something better is needed.

I talked about this with other developers who actually suggested I redo the whole system (as you're going to do), but I couldn't do that because it would take too much time (which, for me, was already limited back then).

chrisu wrote:

So any plugin should only do things and transfer/use data that it really needs to. Second thing is that anyone uses her favorite programming language and there are so much things that are different between those all that the interface needs to be generic as can be. A typical API-Dll as it's usually done should do this job nicely.


You kinda answered my question about what the API actually is here. So it's a plugin, which means it's a DLL. Other plugins will need to reference this dll. How does this affect languages used? It shouldn't be much of a problem using C++ and Delphi to create plugins. Any other languages you want to support? What about .NET languages?

I'll be on a ski-holiday next week (Austria Wink) so I won't be able to comment then. But keep this discussion alive, I'll be back soon. Smile
_________________
Check out Figuro, our online 3D app! More powerful 3D tools for free.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
chrisu
DeleD PRO user


Joined: 11 Sep 2009
Posts: 53
Location: Austria

PostPosted: Mon Jan 10, 2011 9:32 am    Post subject: Reply with quote

Quote:
Would that API be a DLL?

Yes

Quote:
Is there any reason why things couldn't be handled real-time?

Not for normal plugins but for plugins that are acting kinda standalone and therefore not "on-demand". They would need to request or send data as they want. But we can also define a special-event for that reason. Hm, maybe a solution.

Quote:
It would be nice if a plugin could nest itself within the existing Geometry Toolbox. Any thoughts on that?

Yes, I've planned it that way but I have no details on that this time.

Quote:
The reason I choose to pass the XML trees, is because I intended plugins to be used for import/export uses only, in which case passing the entire structure is quite alright.

Understood and agreed. But I don't know if the new interface for import/export will stay the same. However, the old plugins won't be affected by this as the old interface will be supported, too. But it's ought to be understood as being deprecated.

Quote:
How does this affect languages used? It shouldn't be much of a problem using C++ and Delphi to create plugins. Any other languages you want to support? What about .NET languages?

Usually, when you choose the DLL-way there shouldn't be any problem for other languages. The DLL-interface will further be defined to work in a procedural way - so, not object-oriented. Because this might interfere with not-object-oriented languages. The only thing that's needed - but only for comfort-reasons - are definition-files (like header-files in C) for any programming-language used, that brings up better checkability for the compiler used.

Cheers,
Chrisu.
Back to top
View user's profile Send private message Visit poster's website
Starnick
DeleD PRO user


Joined: 28 Jul 2007
Posts: 611

PostPosted: Mon Jan 10, 2011 11:34 pm    Post subject: Reply with quote

Heh this sounds a lot like what I've been reading about with SlimDX - "export interop". There's a danger when exporting objects between libraries (especially unmanaged objects to a managed environment), since the data can change without your control at anytime. You can't cache anything that isn't invariant. It certainly would be doable, but probably more trouble than what's it worth. A few DLL exports that a C# plugin (or other .NET languages) could use via P/Invoke would work just fine.

This API sounds more like a layer between DeleD and a plugin where the plugin hooks into the API, and the API hooks into DeleD (basically events and callbacks). Rather than some sort of job-queue thing.

E.g. For a registered plugin that adds a new tool to the geometry tools, when deled loads the plugin, the plugin sets a hook to the API. When deled sets up its geometry toolbox, it asks the API's for any registered tools, the callback gets activated, pertinent data gets sent across. Whenever the user uses said tool, the plugin gets notified and performs the job.
Back to top
View user's profile Send private message Visit poster's website AIM Address
chronozphere
DeleD PRO user


Joined: 20 Jun 2006
Posts: 1010
Location: Netherlands

PostPosted: Tue Jan 11, 2011 8:46 pm    Post subject: Reply with quote

Hey dudez!

I'm glad I noticed this discussion here. Smile

I didn't read the whole thread, but I noticed the diagram. It looks pretty good to me. Nice separation of modules. Go for it!

I do have one concern. Me and Paul Nicholls have been working on our paths too, but we didnt finish it. I hope the svn repository is in "usable" state. If not, someone could reverse some of our changes, or make a new branch.

If I understand this right, you will make this new interface work as a plugin into the existing interface?
Or will you make a new plugin-interface and create an "adapter" DLL for backward-compatibility?

I'd encourage you to take a good look at the deled source before planning alot of features. Make a small start by introducing a few functions to send/recieve vertices and expand the API from there.
It's tempting to do a top-down design, but when you do that, you will realize more that it will take a long time before it's finished.
Starting at the bottom and making decisions as you move forward is likely to be a more satisfying method. Wink

If you have some questions about the DeleD codebase or Delphi in general, don't hessitate to contact me. I may be able to assist you there. Smile

Good luck!
Back to top
View user's profile Send private message
chrisu
DeleD PRO user


Joined: 11 Sep 2009
Posts: 53
Location: Austria

PostPosted: Tue Jan 11, 2011 10:43 pm    Post subject: Reply with quote

@Starnick: I've thrown the queue-thingy overboard. It's also doable without them and your general concerns about them where right. However, it was just an initial idea.

@chronozphere: I didn't touch the codebase yet, I just checked out the code as described in the WIKI with the plan to generate a package (or patch ... can't remember) which I'll give to you. If that aint right please gimme a note.

The old plugins will continue to work and it will also be possible to use the old interface - but without API. The API itself will be a "simple" DLL whose exported functions will be wellknown by DeleD. DeleD will therefore have a very specifical, "private" communication with this DLL. Any new plugin will initially get connected with DeleD and the API-DLL. So some kind of triangle-relationship Wink I only hope none of those is gettn jelous. Smile
Thx for your advices, I appreciate them.

Cheers,
Chrisu.
Back to top
View user's profile Send private message Visit poster's website
chrisu
DeleD PRO user


Joined: 11 Sep 2009
Posts: 53
Location: Austria

PostPosted: Wed Jan 12, 2011 3:55 pm    Post subject: Reply with quote

I made a pretty detailed sketch today which focuses the components and all necessary stuff of the new plugin-concept and it occurred to me that there might be also a way without the need for an extra-dll for the upcoming API. It should also be possible to implement the API-interface directly in DeleD. I think that this new evolution would make the whole concept much easier to handle and reduces complexity. I'm rather inclined to do it that "new" way. Any concernes?
I'll try to get the (updated) concept-sketch done as an image which I can post here. However, having this technical overview now gives me a positive feeling according to its "do-ability".

Cheers,
Chrisu.
Back to top
View user's profile Send private message Visit poster's website
Starnick
DeleD PRO user


Joined: 28 Jul 2007
Posts: 611

PostPosted: Wed Jan 12, 2011 7:24 pm    Post subject: Reply with quote

chrisu wrote:
I made a pretty detailed sketch today which focuses the components and all necessary stuff of the new plugin-concept and it occurred to me that there might be also a way without the need for an extra-dll for the upcoming API. It should also be possible to implement the API-interface directly in DeleD. I think that this new evolution would make the whole concept much easier to handle and reduces complexity. I'm rather inclined to do it that "new" way. Any concernes?
I'll try to get the (updated) concept-sketch done as an image which I can post here. However, having this technical overview now gives me a positive feeling according to its "do-ability".

Cheers,
Chrisu.


When I first commented above that thought crossed my mind, because then you'd have less to manage. But on the other hand, if you define a very generic flow of data between the API and DeleD you could have the two vary. So rather than recompiling DeleD and distributing, you just have to recompile and distribute the API. Certainly would be easier collapsing it all into the exe though. Less chance of error too.
Back to top
View user's profile Send private message Visit poster's website AIM Address
chrisu
DeleD PRO user


Joined: 11 Sep 2009
Posts: 53
Location: Austria

PostPosted: Wed Jan 12, 2011 8:37 pm    Post subject: Reply with quote

Starnick wrote:
So rather than recompiling DeleD and distributing, you just have to recompile and distribute the API.

Yup, also had this in mind but on the other hand when I modify the API-dll it's mostly probable that you have to do the same or nearly on DeleD's side cuz the DLL would just be a "mask" for DeleD. Anyway, as I mentioned it's a rather easy decision I guess which imho leads to "no DLL". Wink
Back to top
View user's profile Send private message Visit poster's website
chrisu
DeleD PRO user


Joined: 11 Sep 2009
Posts: 53
Location: Austria

PostPosted: Mon Jan 17, 2011 11:13 pm    Post subject: Reply with quote

Here's the big picture of the brand new plugin concept:



I hope it's kinda understandable. Wink
Feel free to ask!

Cheers,
Chrisu.
Back to top
View user's profile Send private message Visit poster's website
chronozphere
DeleD PRO user


Joined: 20 Jun 2006
Posts: 1010
Location: Netherlands

PostPosted: Tue Jan 18, 2011 5:37 pm    Post subject: Reply with quote

Nice one! Smile

Which program did you use to make this diagram. It looks really clean!

I like the way you pass an entire API to the plugin by using proc/method pointers. You could either pass an array of pointers and cast/assign them to variables in the plugin API file. You could also go for a record that contains the right methods.

The first approach is probably best because it is extendable. If you want to add an extra method to the API, just make the array bigger. If you choose for records, you will have problems when the plugin API changes.
Maybe you can use version numbers to check if the plugin and the plugin-interface are of the same version?

Also, Make sure that all those methods sufficiently check for errors. We don't want the plugin to mess up the internal state of deled.

Looking good. Keep up the good work! Smile
Back to top
View user's profile Send private message
AWM Mars
Member


Joined: 06 Jan 2010
Posts: 1195
Location: Wilts England

PostPosted: Tue Jan 18, 2011 6:07 pm    Post subject: Reply with quote

chronozphere wrote:
Nice one! Smile
Which program did you use to make this diagram. It looks really clean!

My guess is, it can be done easily with MS Word, or simply something like Fireworks (my personal favourite). Cool
_________________
Politeness is priceless when received, cost nothing to own or give, yet some cannot afford.

Checkout:
http://www.awm.mars.yourinside.com/
http://www.bccservices.co.uk
http://www.localtradecheck.co.uk
Back to top
View user's profile Send private message Visit poster's website
chrisu
DeleD PRO user


Joined: 11 Sep 2009
Posts: 53
Location: Austria

PostPosted: Tue Jan 18, 2011 9:11 pm    Post subject: Reply with quote

I've done the diagram with good old Visio.
The pointerarray is imho in many ways a good solution. You can i.e. initially make it 1000 elements huge, though there aren't so many API-functions. As the array is only transferred once (init) there's no big deal when it comes to performance. But that way it's rather easy to keep backward-compatibility without the need to re-compile any plugin when something changes in DeleD. If a function gets deprecated you just define the new one in the next free slot and the old one stays as long as needed keeping old plugins still functioning. An oversized array also has huge puffer-abilities - no grow -> no need to re-compile the dependencies.

One thing worth to mention. The EventRegistrator will get one or two arguments added to make it possible for a plugin to define some kind of call-intervall so that DeleD doesn't have to serve a i.e. viewer-plugin permanently with data that doesn't change that often.

The real brains is to define the list of events (though it's no problem to have that growing dynamically). So for that specific list we should do some sort of public brainstorming between us developers. Also we should create a list of API-functionality ordered according to their priority. Again something we might do together.

Thoughts?

Cheers,
Chrisu.
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies.    DeleD Community Edition Forum Index -> DeleD Community Edition All times are GMT
Goto page Previous  1, 2, 3  Next
Page 2 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum