Tag Archives: Transcriptive

Dumb A.I., Dumb Anarchist: Using the Transcriptive Glossary

We’ve been working on Transcriptive for like 3 years now. In that time, the A.I. has heard my voice saying ‘Digital Anarchy’ umpteen million times. So, you would think it would easily get that right by now. As the below transcript from our SRT Importing tutorial shows… not so much. (Dugal Accusatorial? Seriously?)

ALSO, you would think that by now I would have a list of terms that I would copy/paste into Transcriptive’s Glossary field every time I get a transcript for a tutorial. The glossary helps the A.I. determine what  ‘vocal sounds’ should be when it translates those sounds into words. Uh, yeah… not so much.

So… don’t be like AnarchyJim. If you have words you know the A.I. probably won’t get: company names, industry jargon, difficult proper names (cool blog post on applying player names to an MLB video here), etc., then use Transcriptive’s glossary (in the Transcribe dialog). It does work. (and somebody should mention that to the guy that designed the product. Oy.)

Use the Glossary field in the Transcribe dialog!Overall the A.I. is really accurate and does usually get ‘Digital Anarchy’ correct. So I get lazy about using the glossary. It is a really useful thing…

A.I. Glossary in Transcriptive

Importing an SRT into Premiere Pro 2020 & 2021

(The above video covers all this as well, but for those who’d rather read, than watch a video… here ya go!)

Getting an SRT file into Premiere is easy!

But, then it gets not so easy getting it to display correctly.

This is mostly fixed in the new caption system that Premiere 2021 has. We’ll go over that in a minute, but first let’s talk about how it works in Premiere Pro 2020. (if you only care about 2021, then jump ahead)

Premiere Pro 2020 SRT Import

1: Like you would import any other file, go to File>Import or Command/Control+I.

2: Select the SRT file you want.

3: It’ll appear in your Project panel.

4: You can drag it onto your timeline as you would any other file.

Now the fun starts.

Enable Captions from the Tool menu
5: From the Tools menu in the Program panel (the wrench icon), make sure Closed Captions are enabled.

5b: Go into Settings and select Open Captions

6: The captions should now display in your Program panel.

7: In many cases, SRT files start off being displayed very small.

You're gonna need bigger captionsThose bigger captions sure look good!

8: USUALLY the easiest way to fix this is to go to the Caption panel and change the point size. You do this by Right+Clicking on any caption and ‘Select All’. (this is the only way you can select all the captions)

Select all the captions

8b: With all the captions selected, you can then change the Size for all of them. (or change any other attribute for that matter)

9: The other problem that occurs is that Premiere will bring in an SRT file with a 720×486 resoltion. Not helpful for a 1080p project. In the lower left corner of the Caption panel you’ll see Import Settings. Click that to make sure it matches your Project settings.

Import settings for captions

Other Fun Tricks: SRTs with Non-Zero Start Times

If your video has an opening without any dialog, your SRT file will usually start with a timecode other than Zero. However, Premiere doesn’t recognize SRTs with non-zero start times. It assumes ALL SRT files start at zero. If yours does not, as in the example below, you will have to move it to match the start of the dialog.

You don’t have to do this with SRTs from Transcriptive. Since we know you’re likely using it in Premiere, we add some padding to the beginning to import it correctly.

Premiere doesn't align the captions wtih the audioIf your captions start at 05:00, Premiere puts them at 00:00

Importing an SRT file in Premiere 2021: The New Caption System!

(as of this writing, I’m using the beta. You can download the beta by going to the Beta section of Creative Cloud.)

0: If you’re using the beta, you need to enable this feature from the Beta menu. Click it on it and ‘Enable New Captions’.

1: Like you would import any other file, go to File>Import or Command/Control+I.

2: Select the SRT file you want.

3: It’ll appear in your Project panel.

4: You can drag it onto your timeline as you would any other file… BUT

This is where things get different!

4b: Premiere 2021 adds it to a new caption track above the normal timeline. You do need to tell Premiere you want to treat them as Open Captions (or you can select a different option as well)

4c: And Lo! It comes in properly sized! Very exciting.

5: There is no longer a Caption panel. If you want to edit the text of the captions, you need to select the new Text panel (Windows>Text). There you can edit the text, add new captions, etc.

6: To change the look/style of the captions you now need to use the Essential Graphics panel. There you can change the font, size, and other attributes.

Overall it’s a much better captions workflow. So far, from what I’ve seen it works pretty well. But I haven’t used it much. As of this writing it’s still in beta and regardless there may be some quirks that show up with heavier use. But for now it looks quite good.

Transcriptive and the new Adobe Captions

As you’ve probably heard, Adobe announced a new caption system a few weeks ago. We’ve been fielding a bunch of questions about it and how it affects Transcriptive, so I figured I’d let y’all know what our take on it is, given what we know.

Overall it seems like a great improvement to how Premiere handles captions. Adobe is pretty focused on captions. So that’s mainly what the new system is designed to deal with and it looks impressive. While there is some overlap with Transcriptive in regards to editing the transcript/captions, as far as we can tell there isn’t really anything to help you edit video. And there’s a lot of functionality in Transcriptive that’s designed to help you do that. As such, we’re focused on enhancing those features and adding to that part of the product.

Also, it also looks like it’s only going to work with sequences. It _seems_ that when they add the speech-to-text (it’s not available in the beta yet), it’s mostly designed for generating captions for the final edit.

However, being able to transcribe clips and use the transcript to search the clip in the Source panel is one powerful feature that Transcriptive will allow you to do. You can even set in/out points in Transcriptive and then drop that cut into your main sequence.

The ability to send the transcript to a client/AE that doesn’t have Premiere and let them edit it in a web browser is another.

With Transcriptive’s Conform feature, you can take the edited transcript and use it as a Paper Cut. Conform will build a sequence with all the edits.

Along with a bunch of other smaller features, like the ability to add Comments to the transcript.

So… we feel there will still be a lot of value even once the caption system is released. If we didn’t… we would’ve stopped development on it. But we’re still adding features to it… v2.5.1, which lets you add Comments to the transcript, is coming out this week sometime (Dec. 10th, give or take).

One thing we do know, is that the caption system will only import/export caption files (i.e. SRT, SCC, etc). From our perspective, this is not a smart design. It’s one of my annoyances with the current caption system. Transcriptive users have to export a caption file and re-import that into Premiere. It’s not a good workflow, especially when we should just be able to save captions directly to your timeline. Adobe is telling us it’s going to be the same klugy workflow.

So if that doesn’t sound great to you, you can go to the Adobe site and leave a comment asking for JSON import/export. (URL: https://tinyurl.com/y4hofqoa) Perhaps if they hear from enough people, they’ll add that.

Why would that help us (and you)? When we get a transcript back from the A.I., it’s a rich-data text file (JSON format). It has a lot of information about the words in it. Caption formats are data poor. It’s kind of like comparing a JPEG to a RAW file. You usually lose a lot of information when you save as a caption format (as you do with a JPEG).

It’ll make it much easier for us and other developers to move data back and forth between the caption system and other tools. For example: If you want someone to make corrections to the Adobe transcript outside of Premiere (on Transcriptive.com for example :-), it’s easier to keep the per-word timecode and metadata with a JSON file.

Historically Adobe has had products that were very open. It’s why they have such a robust plugin/third-party ecosystem. So we’re hopeful they continue that by making it easy to access high resolution data from within the caption system or anywhere else data/metadata is being generated.

It’s great Adobe is adding a better caption workflow or speech-to-text. The main reason Transcriptive isn’t more caption-centric is we knew Adobe was going to upgrade that sooner or later. But the lack of easy import/export is a bummer. It really doesn’t help us (or any developer) extend the caption system or help Premiere users that want to use another product in conjunction with the system. As mentioned, it’s still beta, so we’ll see what happens. Hopefully they make it a bit more flexible and open.

Cheers,
Jim Tierney
Chief Executive Anarchist

Improving Accuracy of A.I. Transcripts with Custom Vocabulary

The Glossary feature in Transcriptive is one way of increasing the accuracy of the transcripts generated by artificial intelligence services. The A.I. services can struggle with names of people or companies and it’s a big of mixed bag with technical terms or industry jargon. If you have a video with names/words you think the A.I. will have a tough time with, you can enter them into the Glossary field to help the A.I. along.

For example, I grabbed this video of MLB’s top 30 draft picks in 2018:

Obviously a lot names that need to be accurate and since we know what they are, we can enter them into the Glossary.

Transcriptive's Glossary to add custom vocabulary

As the A.I. creates the transcript, words that sound similar to the names will usually be replaced with the Glossary terms. As always, the A.I. analyzes the sentence structure and makes a call on whether the word it initially came up with fits better in the sentence. So if the Glossary term is ‘Bohm’ and the sentence is ‘I was using a boom microphone’, it probably won’t replace the word. However if the sentence is ‘The pick is Alex boom’, it will replace it. As the word ‘boom’ makes no sense in that sentence.

Here are the resulting transcripts as text files: Using the Glossary and Normal without Glossary

Here’s a short sample to give you an idea of the difference. Again, all we did was add in the last names to the Glossary (Mize, Bart, Bohm):

With the Glossary:

The Detroit Tigers select Casey Mize, a right handed pitcher. From Auburn University in Auburn, Alabama. With the second selection of the 2018 MLB draft, the San Francisco Giants select Joey Bart a catcher. A catcher from Georgia Tech in Atlanta, Georgia, with the third selection of a 2018 MLB draft. The Philadelphia Phillies select Alec Bohm, third baseman

Without the Glossary:

The Detroit Tigers select Casey Mys, a right handed pitcher. From Auburn University in Auburn, Alabama. With the second selection of the 2018 MLB draft, the San Francisco Giants select Joey Bahrke, a catcher. A catcher from Georgia Tech in Atlanta, Georgia, with the third selection of a 2018 MLB draft. The Philadelphia Phillies select Alec Bomb. A third baseman

As you can see it corrected the names it should have. If you have names or words that are repeated often in your video, the Glossary can really save you a lot of time fixing the transcript after you get it back. It can really improve the accuracy, so I recommend testing it out for yourself!

It’s also worth trying both Speechmatics and Transcriptive-A.I. Both are improved by the glossary, however Speechmatics seems to be a bit better with glossary words. Since Transcriptive-A.I. has a bit better accuracy normally, you’ll have to run a test or two to see which will work best for your video footage.

If you have any questions, feel free to hit us up at cs@nulldigitalanarchy.com!

Transcriptive and 14.x: Why New World Needs to be Off

Update: For Premiere 14.3 and above this is no longer the case. New World is working sufficiently well at this point that Transcriptive seems to be working well with it. Adobe has fixed various bugs with it and things are working as they should.

We’re keeping an eye on it as more people use it, but for now it seems good to go. However, if you start running into weird errors you might want to try turning New World off to see if that makes a difference.

Original Post:

There are a variety of problems with Adobe’s new Javascript engine (dubbed New World) that’s part of 14.0.2 and above. Transcriptive 2.0 will now automatically turn it off and you’ll need to restart Premiere. Transcriptive 2.0 will not work otherwise.

If you’re using Transcriptive v1.5.2, please see this blog post for instructions on turning it off manually.

For the most part Transcriptive, our plugin for transcribing in Premiere, is written in Javascript. This relies on Premiere’s ability to process and run that code. In Premiere 14.0.x, Adobe has quietly replaced the very old Extendscript interpreter with a more modern Javascript engine (It’s called ‘NewWorld’ in Adobe parlance and you can read more about it and some of the tech-y details on the Adobe Developer Blog). On the whole, this is a good thing.

However, for any plugin using Javascript, it’s a big, big deal. And, unfortunately, it’s a big, big deal for Transcriptive. There are a number of problems with it that, as of 14.1, break both old and new versions of Transcriptive.

As with most new systems, Adobe fixes a bunch of stuff and breaks a few new things. So we’re hoping over the next couple months they work all the kinks out and it all sorts itself out.

There is no downside to turning New World off at this point. Both the old and new Javascript engines are in Premiere, so it’s not a big deal as of now. Eventually they will remove the old one, but we’re not expecting that to happen any time soon.

As always, we will keep you updated.

Fwiw, here’s what you’ll see in Transcriptive if you open it with New World turned on:

Premiere needs to be restarted in order to use TranscriptiveThat message can only be closed by restarting Premiere. If New World is on, Transcriptive isn’t usable. So you _must_ restart.

What we’re doing in the background is setting a flag to off. You can see this by pulling up the Debug Console in Premiere. Use Command+F12 (mac) or Control+F12 (windows) to bring up the console and choose Debug Database from the hamburger menu.

You’ll see this:

New World flag set to OffIf you want to turn it back on at some point, this is where you’ll find it. However, as mentioned, there’s no disadvantage to having it off and if you have it on, Transcriptive won’t run.

If you have any questions, please reach out to us at cs@nulldigitalanarchy.com.

Transcriptive: Here’s how to transcribe using your Speechmatics credits for now.

If you’ve been using Speechmatics credits to transcribe in Transcriptive, our transcription plugin for Premiere Pro, then you noticed that accessing your credits in Transcriptive 2.0.2 and later is not an option anymore. Speechmatics is discontinuing the API that we used to support their service in Transcriptive, which means your Speechmatics credentials can no longer be validated inside of the Transcriptive panel.

We know a lot of users still have Speechmatics credits and have been working closely with Speechmatics so those credits can be available in your Transcriptive account as soon as possible. Hopefully in the next week or two.

In the meantime, there are a couple ways users can still transcribe with Speechmatics credits. 1) Use an older version of Transcriptive like v1.5.2 or v2.0.1. Those should still work for a bit longer but uses the older, less accurate API or 2) Upload directly on their website and export the transcript as a JSON file to be imported into Transcriptive.  It is a fairly simple process and a great temporary solution for this. Here’s a step-by-step guide:

1. Head to the Speechmatics website – To use your Speechmatics credits, head to www.speechmatics.com and login to your account. Under “What do you want to do?”, choose “Transcription” and select the language of your file. 

Speechmatics_Uploading

2. Upload your media file to the Speechmatics website – Speechmatics will give you the option to drag and drop or select your media from a folder on your computer. Choose whatever option works best for you and then click on “Upload”. After the file is uploaded, the transcription will start automatically and you can check the status of the transcription on your “Jobs” list.  
Speechmatics_Transcribing3. Download a .JSON file –  After the transcription is finished (refresh the page if the status doesn’t change automatically!), click on the Actions icon to access the transcript. You will then have the option to export the transcript as a .JSON file

Speechmatics_JSON

4. Import the .JSON file into any version of Transcriptive – Open your Transcriptive panel in Premiere. If you are usingTranscriptive 2.0,  be sure Clip Mode is turned on. Select the clip you have just transcribed on Speechmatics and click on “Import”.  If you are using an older version of Transcriptive, drop the clip into a sequence before choosing “Import”. 

Transcriptive_Import

You will then have the option to “Choose an Importer”. Select the JSON option and import the Speechmatics file saved on your computer. The transcript will be synced with the clip automatically at no additional charge.

Transcriptive_Json

One important thing to know is that, although Transcriptive v1.x still have Speechmatics as an option and it still works, we would still recommend following the steps above to transcribe with Speechmatics credits. The option available in these versions of the panel is an older version of their API and less accurate than the new version. So we recommend you transcribe on the Speechmatics website if you want to use your Speechmatics credits now and not wait for them to be transferred.

However, we should have the transfer sorted out very soon, so keep an eye open for an email about it if you have Speechmatics credits. If the email address you use for Speechmatics is different than the one you use for Transcriptive.com, please email cs@nulldigitalanarchy.com. We want to make sure we get things synced up so the credits go to the right place!

Adobe Premiere 14.0.2 and Transcriptive: What You Need to Know

Adobe has slipped in a pretty huge change into 14.0.2 and it seriously affects Transcriptive, the A.I. transcript plugin for Premiere. I’ll get into the details in a moment, but let me get into the important stuff right off the bat:

  • If you are using Premiere 14.0.2 (the latest release)
    • And own Transcriptive 2.0…
    • And own Transcriptive 1.x…
      • You can upgrade to Transcriptive 2.x
      • Or you must turn ‘NewWorld’ off (instructions are below)
      • Or keep using Premiere Pro 14.0.1

For the most part Transcriptive is written in Javascript. This relies on Premiere’s ability to process and run that code. In Premiere 14.0.2, Adobe has quietly replaced the very old Extendscript interpreter with a more modern Javascript engine (It’s called ‘NewWorld’ in Adobe parlance and you can read more about it and some of the tech-y details on the Adobe Developer Blog). On the whole, this is a good thing.

However, for any plugin using Javascript, it’s a big, big deal. And, unfortunately, it’s a big, big deal for Transcriptive. It completely breaks old versions of Transcriptive.

If you’re running Transcriptive 2.x, no problem… we just released v2.0.3 which should work fine with both old and new Javascript Interpreter/engine.

If you’re using Transcriptive 1.x, it’s still not exactly a problem but does require some hoop jumping. (and eventually ‘Old World’ will not be supported in Premiere and you’ll be forced to upgrade TS. That’s a ways off, though.)

Turning Off New World

Here are the steps to turn off ‘NewWorld’ and have Premiere revert back to using ‘Old World’:

  • Press Control + F12 or Command + F12. This will bring up Premiere’s Console.
  • From the Hamburger menu (three lines next to the word ‘Console’), select Debug Database View
  • Scroll down to ScriptLayerPPro.EnableNewWorld and uncheck the box (setting it to False).
  • Restart Premiere Pro

When Premiere restarts, NewWorld will be off and Transcriptive 1.x should work normally.

Screenshot of Premiere's Debug console
So far there are no new major bugs and relatively few minor ones that we’re aware of when using Transcriptive 2.0.3 with Premiere 14.0.2 (with NewWorld=On). There are also a LOT of other improvements in 2.0.3 that have nothing to do with this.

Adobe actually gave us a pretty good heads up on this. Of course, in true Anarchist fashion, we tested it early on (and things were fine) and then we tested it last week and things were not fine. So it’s been an interesting week and a half scrambling to make sure everything was working by the time Adobe sent 14.0.2 out into the world.

So everything seems to be working well at this point. And if they aren’t, you now know how to turn off all the new fangled stuff until we get our shit together! (but we do actually think things are in good shape)

Your transcripts are out of order! This whole timeline’s out of order!

When cutting together a documentary (or pretty much anything, to be honest), you don’t usually have just a single clip. Usually there are different clips, and different portions of those clips, here, there and everywhere.

Our transcription plugin, Transcriptive, is pretty smart about handling all this. So in this blog post we’ll explain what happens if you have total chaos on your timeline with cuts and clips scattered about willy nilly.

If you have something like this:

Premiere Pro Timeline with multiple clips
Transcriptive will only transcribe the portions of the clips necessary. Even if the clips are out of order. For example, the ‘Drinks1920’ clip at the beginning might be a cut from the end of the actual clip (let’s say 1:30:00 to 1:50:00) and the  Drinks cut at the end might be from the beginning (e.g. 00:10:00 to 00:25:00).

If you transcribe the above timeline, only 10:00-25:00 and 1:30:00-1:50:00 of Drinks1920.mov will be transcribed.

If you Export>Speech Analysis, select the Drinks clip, and then look in the Metadata panel, you’ll see the Speech Analysis for the Drinks clip will have the transcript for those portions of the clip. If you drop those segments of the Drinks clip into any other project, the transcript comes along with it!

The downside to _only_ transcribing the portion of the clip on the timeline is, of course, the entire clip doesn’t get transcribed. Not a problem for this project and this timeline, but if you want to use the Drinks clip in a different project, the segment you choose to use (say 00:30:00 to 00:50:00) may not be previously transcribed.

If you want the entire clip transcribed, we recommend using Batch Transcribe.

However, if you drop the clip into another sequence, transcribe a time span that wasn’t previously transcribed and then Export>Speech Analysis, that new transcription will be added to the clips metadata. It wasn’t always this way, so make sure you’re using Transcriptive v1.5.2.  If you’re in a previous version of Transcriptive and you Export>Speech Analysis to a clip that already has part of a transcript in SA, it’ll overwrite any transcripts already there.

So feel free to order your clips any way you want. Transcriptive will make sure all the transcript data gets put into the right places. AND… make sure to Export>Speech Analysis. This will ensure that the metadata is saved with the clip, not just your project.

Artificial Intelligence is The New VR

Couple things stood out to me at NAB.

1) Practically every company exhibiting was talking about A.I.-something.

2) VR seemed to have disappeared from vendor booths.

The last couple years at NAB, VR was everywhere. The Dell booth had a VR simulator, Intel had a VR simulator, booths had Oculuses galore and you could walk away with an armful of cardboard glasses… this year, not so much. Was it there? Sure, but it was hardly to be seen in booths. It felt like the year 3D died. There was a pavilion, there were sessions, but nobody on the show floor was making a big deal about it.

In contrast, it seemed like every vendor was trying to attach A.I. to their name, whether they had an A.I. product or not. Not to mention, Google, Amazon, Microsoft, IBM, Speechmatics and every other big vendor of A.I. cloud services having large booths touting how their A.I. was going to change video production forever.

I’ve talked before about the limitations of A.I. and I think a lot of what was talked about at NAB was really over promising what A.I. can do. We spent most of the six months after releasing Transcriptive 1.0 developing non-A.I. features to help make the A.I. portion of the product more useful. The release were announcing today and the next release coming later this month will focus on getting around A.I. transcripts completely by importing human transcripts.

There’s a lot of value in A.I. It’s an important part of Transcriptive and for a lot use cases it’s awesome. There are just also a lot of limitations.  It’s pretty common that you run into the A.I. equivalent of the Uncanny Valley (a CG character that looks *almost* human but ends up looking unnatural and creepy), where A.I. gets you 95% of the way there but it’s more work than it’s worth to get the final 5%. It’s better to just not use it.

You just have to understand when that 95% makes your life dramatically easier and when it’s like running into a brick wall. Part of my goal, both as a product designer and just talking about it, is to help folks understand where that line in the A.I. sand is.

I also don’t buy into this idea that A.I. is on an exponential curve and it’s just going to get endlessly better, obeying Moore’s law like the speed of processors.

When we first launched Transcriptive, we felt it would replace transcriptionists. We’ve been disabused of that notion. ;-) The reality is that A.I. is making transcriptionists more efficient. Just as we’ve found Transcriptive to be making video editors more efficient. We had a lot of folks coming up to us at NAB this year telling us exactly that. (It was really nice to hear. :-)

However, much of the effectiveness of Transcriptive comes more from the tools that we’ve built around the A.I. portion of the product. Those tools can work with transcripts and metadata regardless of whether they’re A.I. or human generated. So while we’re going to continue to improve what you can do with A.I., we’re also supporting other workflows.

Over the next couple months you’re going to see a lot of announcements about Transcriptive. Our goal is to leverage the parts of A.I. that really work for video production by building tools and features that amplify those strengths, like PowerSearch our new panel for searching all the metadata in your Premiere project, and build bridges to other technology that works better in other areas, such as importing human created transcripts.

Should be a fun couple months, stay tuned! btw… if you’re interested in joining the PowerSearch beta, just email us at cs@nulldigitalanarchy.com.

Addendum: Just to be clear, in one way A.I. is definitely NOT VR. It’s actually useful. A.I. has a lot of potential to really change video production, it’s just a bit over-hyped right now. We, like some other companies, are trying to find the best way to incorporate it into our products because once that is figured out, it’s likely to make editors much more efficient and eliminate some tasks that are total drudgery. OTOH, VR is a parlor trick that, other than some very niche uses, is going to go the way of 3D TV and won’t change anything.

Jim Tierney
Chief Executive Anarchist
Digital Anarchy

Getting transcripts for Premiere Multicam Sequences

Using Transcriptive with multicam sequences is not a smooth process and doesn’t really work. It’s something we’re working on coming up with a solution for but it’s tricky due to Premiere’s limitations.

However, while we sort that out, here’s a workaround that is pretty easy to implement. Here are the steps:

1- Take the clip with the best audio and drop it into it’s own sequence.
Using A.I. to transcribe Premiere Multicam Sequences
2- Transcribe that sequence with Transcriptive.
3- Now replace that clip with the multicam clip.
Transcribing multicam in Adobe premiere pro

4- Voila! You have a multicam sequence with a transcript. Edit the transcript and clip as you normally would.

This is not a permanent solution and we hope to make it much more automatic to deal with Premiere’s multicam clips. In the meantime, this technique will let you get transcripts for multicam clips.

Thanks to Todd Drezner at Cohn Creative for suggesting this workaround.