As part of an on-going discussion on the KiCAD devleopers mailing list about library crowdsourcing, I wanted to put together more detail about the issue. I've been using KiCAD for a handful of months and was unaware of some of the organizations, such as CircuitHub and KiCadCloud, that have been working to improve the situation until this discussion.
CircuitHub uses a novel approach. By combining their site with DropBox, circuithub will add to your DropBox the parts you've selected via the web interface. By using the filesystem and not attempting to interface with the application directly, CircuitHub is able to interface with Altium, KiCAD and EagleCAD.
KiCadCloud.com allows users to download parts libraries as well as upload them if they've created an account on the site.
Kicadlib.org is a site where users can have their libraries added as links from a main page. There are quite a few parts available through the libraries. New libraries appear to be submitted via email to the maintainer.
If the part you are looking for isn't available through any of those online services, you'll have to create it yourself. Creating parts can be time consuming and error prone. How can we improve this situation by adding value to the user?
Background
PCB development flow
A typical circuit board development process in KiCAD looks something like:Schematic (eeschema)
- Design a circuit
- Locate components (or create the component symbols)
- Add components to the schematic
- Connect components on a schematic using the component symbol
Map component to footprint (cvpcb)
- Choose the specific footprint for each component in the schematic (or create the necessary footprints)
Layout (pcbnew)
- Place components
- Route the signals on the board
The problem
During schematic step #2 and the component mapping process, the highlighted steps above, any component that doesn't already exist has to either be created or downloaded. While there are new parts being introduced regularly there are also a lot of people using KiCAD (and other eda tools) that are creating the same part symbols and footprints.A solution
Make it easier for users to share and use part symbols and footprints so other users could use their part models rather than creating their own.
Current approaches
There are a few different online solutions where you can get or share components.circuithub.com
CircuitHub uses a novel approach. By combining their site with DropBox, circuithub will add to your DropBox the parts you've selected via the web interface. By using the filesystem and not attempting to interface with the application directly, CircuitHub is able to interface with Altium, KiCAD and EagleCAD.
kicadcloud.com
KiCadCloud.com allows users to download parts libraries as well as upload them if they've created an account on the site.
kicadlib.org
Kicadlib.org is a site where users can have their libraries added as links from a main page. There are quite a few parts available through the libraries. New libraries appear to be submitted via email to the maintainer.
If the part you are looking for isn't available through any of those online services, you'll have to create it yourself. Creating parts can be time consuming and error prone. How can we improve this situation by adding value to the user?
What the user wants to be able to do
- Find a well created component symbol
- Find a compatible and correct footprint
- Contribute:
- Feedback on symbols or footprints, maybe voting or text comments or bug reports
- New symbols and footprints
- Modified symbols or footprints
- Store used symbols or footprints in local files for version control, to work without a network connection and to mitigate concerns that symbols or footprints might go away.
- If client/server:
- Ability to interact with the server without an account, in a read-only (non-contributor) mode. Simplifies the initial adoption process.
- Web interface
- Browse and view entries in the library
- Add comments to parts
- Search by keywords
- Receive email notification if someone comments on their part or reports an issue.
- The ability to run their own instance of the server
- Companies could use this to share parts among developers or groups
- Prevents vendor lock-in
Helpful features for server operators
Assuming a client server approach, some features that have been beneficial to server operators are:
- Ability to have roles such a maintainers with permissions to help curate the library
- Could use hard defined roles or a reputation system like the one found on StackOverflow
- Runs on off the shelf modern website software
- Runs on cloud instances, makes deploying new servers easy
- Licensing options per-part
- Ways to flag entries as bad
- Email notifications if parts have been flagged
- External user authentication to remove the necessity to manage user accounts on the server
Communication API vs. Server implementation
The communication api and the server would be treated as separate entities. For instance the api may be jointly developed with the community, CircuitHub, and KiCadCloud's participation. Each of these organizations may have their own sever implementation with different features but they would remain inter-operable through the common api.Implementation notes
- Client/server approach
- RESTful API using json
- Simplifies the server API
- Possible API including:
- login / logout
- part search
- get part by hash
- get list of hashes for user account
- put new part
- put new part version
- put part feedback
- Fully documented API
- Open source server side code, GitHub or other online hosting for the server side code
- Possible cooperation with CircuitHub on the API
- Create an official KiCAD developer blessed master server. Provides a trusted source for users and a pre-installed cloud library to get users started with the approach.
- Hash based data tracking
- Based on how git uses hashes
- Hash provides a unique identifier for each model.
- Hash generated from all data elements that are critical. The model data plus author may be a good start.
- Detect identical parts.
- Version control can use the hash
- Database tables
- Data elements for each model
- Author
- Creation date
- License
- Model content (symbol or footprint)
- 3D model
- Comments
- Datasheet url
- Votes
- Server settings
- User settings settings
- User roles etc
Thoughts on leveraging existing tools
- CircuitHub / Dropbox Approach
- Dropbox is a proprietary tool
- How is part versioning handled?
- Concern for vendor lock-in?
- How is a new part added from the client/tool side? A helpful step-by-step guide on the CircuitHub blog shows server->client data flow.
- Using the file system is more transparent to applications but seems to make the server side more complex while limiting the client side functionality. Overall the approach is simpler but may be less extendable and more work for the end user.
- Could the Dropbox approach be abstracted to use git for the data exchange?
- Leverages an existing well tested tool (git).
- How to handle permissions on pushes?
- Would each user have their own branch?
- How are part changes pushed from client to server?
- How would file naming be handled? Is the naming arbitrary and could be the name of the hash value? Odd/long filenames would make it difficult to interact with the parts library via the filesystem.
- Server could interact with each users branch in response to web interaction.
- Could git be leveraged internally on the server side?
- Use git as a general purpose data store on the server side.
- How to attach things like votes, attributes like datasheets etc via git? A database would seem more natural for that.
- Git could handle versioning but how would we decide where to place a part in the filesystem?
- Could use git's internal data store directly.
Comments
Post a Comment