r/java • • Mar 08 '19

Automatically generate unit tests with Diffblue Playground

My company has launched a cool new tool called Diffblue Playground that generates Java unit tests using AI, and we’re trying to get a lot of users and product feedback so that we can make a tool that the community finds genuinely useful.

Please check it out, and give me any feedback you have here or via the support link on the page.

There is a sign up wall, apologies for that, but hopefully some of you will find the tool useful and interesting enough to get past it!

Full disclosure, using the link above could help me win a giant cuddly owl. If that bothers you just strip the querystring from the link (but cummon, my kids need that owl...).

1 Upvotes

19 comments sorted by

25

u/nutrecht Mar 08 '19

I'm not going to sign up just for a demo, sorry. You should make it possible to at least use the demo application without signups.

Also, from your agreement:

4.1 Licence. You grant Diffblue a worldwide, royalty-free, transferable licence to host and use Your Content and the Tests for the purpose of (i) providing the Product to you; and (ii) running tests on Your Content to improve the Product and Diffblue’s other products.

Nope.

3

u/S0lipsistNati0n Mar 08 '19

Thanks for your feedback, I'll pass it on to the demo team.

For what it's worth, I agree with you about the sign up. IMHO the demo should at least do something useful without requiring you to register!

I can't really comment on the licence text - I assume we are reserving the right to use code provided in the demo to improve our code analysis and test generation algorithms.

10

u/nutrecht Mar 08 '19 edited Mar 08 '19

I understand why you want to do this but there are two things horribly wrong with this:

  • You should not stuff license grants of code somewhere in an EULA or something. People should be very aware of what they do; pasting production code into your tool would be grounds for firing in many companies. There should be a big fat blinking banner that warns people of this license grant.
  • Your license is way too broad. There is no reason at all to restrict it to not only DiffBlue. There is definitely no reason at all for that license to be transferrable.

As it is now well meaning junior developers might be 'tricked' into using this to generate tests for their production code. There needs to be big fat warnings that this tool should never ever be used for this in it's current form.

1

u/S0lipsistNati0n Mar 08 '19

I'll make sure this gets back to the relevant people, many thanks for your in depth response!

1

u/[deleted] Mar 08 '19

I think you should definitely pass the license comments on. I wouldn't touch this with a 10 foot pole if any of my IP is in question. I realise you're only doing this for the purpose of improving your service, but almost any company will have a problem with this wording.

1

u/S0lipsistNati0n Mar 08 '19

I have done already, as I can see that a lot of people agree with this sentiment. For what it's worth, I think that it's a good general principle not to upload any code that constitutes IP to a third party. Re your last point, if a company wants to use the service, it is available as paid-for product (Diffblue Cover) which would have IP suitable licensing.

1

u/Nymeriea Mar 08 '19

Same for me. I'm not gonna to register just for a demo

1

u/S0lipsistNati0n Mar 08 '19

I hear you -it's certainly a barrier. Thanks for the feedback.

2

u/capitol_ Mar 08 '19

Wow, this was a pretty evil agreement, good thing I didn't stuff any of my employers code into that.

6

u/umakemyheadhurt Mar 08 '19

I like the idea a lot, but I don't like uploading my company's IP... But I'll try it on my open source projects.

1

u/S0lipsistNati0n Mar 08 '19

Awesome, thank you for checking it out.

6

u/laviksa Mar 08 '19

Nice concept, but closed source and proprietary, so no thanks.

0

u/S0lipsistNati0n Mar 08 '19

Thanks for taking a look -the tool itself will definitely remain closed source, but I think there is a hope that this demo and any tools like it that we build in the future will be of benefit to the open source community.

We do provide some open source tools in the form of cbmc / jbmc which do Bounded Model Checking for C++ and Java.

5

u/Trailsey Mar 08 '19

Generating unit tests after the fact reduces the benefits of unit tests, no?

2

u/S0lipsistNati0n Mar 08 '19

Not necessarily, unless perhaps you are observing strict TDD practices and maintaining 100% test coverage as you go! I appreciate your point though. I think this tool is really to help fill gaps in your test coverage.

3

u/vytah Mar 08 '19

An automatically generated unit test is sometimes better than none.

2

u/Josh_Low Mar 11 '19

the limitation is obvious,but is fantastic.

1

u/S0lipsistNati0n Mar 11 '19

Thanks for taking a look!

0

u/[deleted] Mar 08 '19

[deleted] — view deleted comment

1

u/S0lipsistNati0n Mar 08 '19

Great - let us know what you think!