Oracle E-Business Suite Views

There's a New Blogger In Town: James Morrow's The DBA Life

2012-02-07T10:30:00.000-08:00

OK, technically, you cannot call James Morrow a new blogger. He set up his blog some time ago. But he did not blog. What a bad boy! The good news is, now he's properly motivated and has started blogging with great guns and plenty of enthusiasm. So take a look:

http://thedbalife.blogspot.com/

Hot topics from James, just off the press this week, include:

Spreadsheet Risk (and why ad-hoc reporting tools make me twitchy)

Adventures in SQL*Net encryption (E-Business Suite edition)

Certification on Oracle (RedHat) Enterprise Linux 6?

Changes are coming to My Oracle Support

More on Includes File from my friend James

2012-01-29T16:57:00.000-08:00

My friend James, who has a very big brain, often tells me additional tidbits about things that I have blogged about. So here again, he has added some additional insight; I hope he won't mind my quoting him:

I've used this for quite some time... particularly useful when you have a concurrent request or a form that performs poorly... you can search using "includes file" in order to figure out if there is simply a newer version of that report/form. Based on the results, you could then (potentially) deduce that maybe there might have been a performance fix that would solve your problem.

Oh my gosh, what an excellent point. James' idea is actually a nicer way of looking for new performance improvements than what I would have done, basically trolling around on My Oracle Support using keywords and such to see if a performance improvement was mentioned for a particularly troublesome report.

Thank you James!

"The keyboard? How quaint."

2012-01-29T12:37:00.000-08:00

I was just over on youtube reviewing that famous Star Trek outtake where Scotty approaches a 20th century computer and tries to get it to respond to his voice. "Computer? Commmputer?" Bones hands Scotty the mouse, and Scotty proceeds to speak into it as if it is a microphone: "Hello Computer..." The computer's owner suggests that Scotty "Just use the keyboard." "The keyboard? How quaint."

I am always reminded of this part of the movie when I get lost on My Oracle Support, trying to find something, and wishing the interface went beyond intuitive and over into psychic. If google can notice that I'm interested in buying a new couch and then spend the next couple of weeks tossing ads at me for furniture dealers, then how far are we from My Oracle Support noticing that I am bobbing around trying to remember how to find something and proactively reminding me how to get there?

Oh, well, in the meantime, I must resort to blogging details about how I managed to do something once so that when I need to again I can search my blog. Today's post will cover Trolling for Patches, and the Ever So Interesting "Includes File".

On MOS, Click on Patches & Updates

Click on Patching Quick Links, and since I am using Release 12.1.3, click on Recommended R12 Patches:

Now, as far as I can tell, it is very important that you click on Advanced Search in this next screen. I tried just muddling around in this screen, and it didn't get me what I was looking for. I changed the Maintenance Release from the default of 12.1.1 to 12.1.3, and I clicked on the flashlight and entered AZ in the screen that popped up, and then I clicked on Select in the popup screen, and it didn't do what I expected. It didn't carry back the AZ selection into the Product or Product Family, so it didn't cut the list down to just AZ patches. I am not sure, but I think that if you haven't clicked on Advanced Search, this is perhaps defaulting to Simple Search, but I am not sure why it would let me click on the Product or Product Family flashlight, choose something, and then not carry it back.

For this example, I wanted to find any patches for AZ, because I am having trouble with iSetup. You can see the fields I filled in.

It is moments like these that make me feel pretty dumb. If I were my 11 year old son, I'd be banging on my desk right now, shouting "Why? Why can't I make it work???"

That said, remember what I said a little earlier, Click on Advanced Search.

It pulled up a better screen, and that screen let me click on a pile of flashlights to limit what I was searching for, which pulled up popup screens that carried back the way I expected them to. And when I clicked on the Go button, it pulled up a nice pile of patches:

Now, I've shown you all this, which solved my problem by giving me a list of AZ patches that I should look at to see if I am missing any that might solve my problem with iSetup, but in the end, that's not the reason I pulled up all this information. Take a look at what I circled above in red: Includes File. That's the thing that I want to remember how to get to. It's terrific to use this screen to get a list of available patches, but the Includes File option is the one that I always forget. Let's say you find a note on My Oracle Support that says that a particular version of a file is needed to solve a problem. Enter it here in the Includes File box, and you can limit the results of your query to just those patches that have that file. That feature is the one I always forget how to use, and so, for posterity, I have recorded it here.

Here's Oracle's definition of Includes File:

This search option is valid only for Applications products. This parameter is not considered while querying for Server or Tools products patches.

This search parameter is used to limit down the search results to only those patches that contain a specific file, or a specific version of a specific file. To use this parameter, the exact, case sensitive, file-name must be known. No wildcards are allowed.

The search can be further limited by setting the file version. No wildcards are allowed for the version field. The search will match all file versions greater than or equal to the selected version.

If you try out Includes File, take heed to what Oracle says above - you have to know exactly what you're looking for, and it doesn't allow wildcards to figure it out.

The end.

Ta.

Da.

A Bevy of Ways to Learn About the E-Business Suite: I've Tried Them All and Got Something Out of All of Them

2012-01-27T14:25:00.000-08:00

I've had a little free time lately, so I've used it well and have been filling my head with interesting things I didn't know before. How's that for good? I'm rather proud of myself. So I thought I'd share some of the places I've been and the things I've learned; all about the E-Business Suite, mind you. This may take more than one blog entry, as my edification has been that good.

Communities > My Oracle Support Community: When I hit a problem with the E-Business Suite, I have a tried and true routine that I follow. Recently I was working on a Release 12 Upgrade Readiness Assessment and visiting a client DBA, and I noticed he does the same thing as me - so it must be good! Basically, I hit an issue and troll My Oracle Support first. If I find a direct hit I dance a happy jig and go with that. If I don't, then I google my problem. Often I find what I need there. And if that doesn't work, I go to http://www.oracle.com/ and choose Communities from the pulldown at the top of the screen, and then My Oracle Support Community. You can search for fellow sufferers of your problem, and if that doesn't work, you can ask a question. Folks on this community go above and beyond the call of duty to be responsive. Be kind to them and tell them as much as you can about your environment when you pose your question. Otherwise they'll have to ask, and you'll have to answer, and that'll slow you down a bit.

OAUG Connection Point regional conferences: I'm biased because they let me work on the Paper Selection Committee, but the OAUG Connection Point conferences are regional conferences, managed by OAUG in partnership with a local user group and an appropriate SIG (Special Interest Group) to cover a hot topic. The first one I participated on was in Chicago in July, 2011. It was just on the Release 12.1 upgrade, and I thought it went very well. There were lots of presentations, all about upgrading. The users were all very focused on the topic. There were also vendor booths and presentations, which made it easy to find consultants and 3rd party solutions to solve some of the gnarlier problems.

The second Connection Point conference that I attended last year was in Atlanta in November. It covered more than just upgrading - the topics were EPM, BI and Release 12 Upgrade. There were lots of great presentations, but the ones I enjoyed most, because I hadn't had a chance to attend them at other conferences, were by Susan Behn, of Infosemantics:

Maximize Efficiency of Oracle EBS Releases 11i/12 through Diagnostics - Learn how Diagnostic tests can be used in your upgrade/implementation to validate configurations, data and much more. These are XML reports that are run from a new, more user friendly interface starting in 12.0.6 and higher. The Diagnostic Tool includes tests to for users in virtually every area of the organization—functional (setups, processes, month end close), technical (analyze data integrity issues, data validation, purge opportunities), DBAs (security best practices, setup.) There is much more.

R12 Function and Data Security- UMX and Role Based Access Control - Release 12 includes stronger controls across the application modules for limiting access to both functions and data. This presentation will discuss these controls and focus on how Oracle User Management (UMX) and the Role Based Access Control (RBAC) standard work to mitigate security risks.

Ten New Forms and OAF Personalization Examples - This is an update to Susan's original popular personalization presentation with ten new examples. As part of your R12 upgrade, use Form and OAF Personalizations to replace many of your customizations. Also use personalization to streamline your business processes through improved control of validations, defaults, terminology and other requirements. These are the newest examples highlighting the newest Release 12.1 features.

Susan's presentation on using Oracle's Diagnostics tool that is part of the Oracle Application Manager offers up one more place where you can root around to figure out what is going on in your E-Business Suite environment. I was tickled to also pick up two more purge programs (I have a soft spot for finding Oracle-supported ways to remove administrative data from an applications environment) - yup, if you're churning out diagnostics reports, you should also be purging the data periodically; check out the Delete Diagnostic Logs and Delete Diagnostic Statistics concurrent programs.

If you're an OAUG member, you can download all of these presentations and more from the OAUG website. And if you're not, then try googling them!

Oracle's Patch Wizard Webinar: Jonathan Janaitis from Oracle gave this webinar about Patch Wizard. I'm a big fan of Patch Wizard. OK, I'm an embarrassingly gushingly enthusiastic fan of Patch Wizard. Jonathan had this one slide that I really liked. I used snagit to grab a picture of it while Jonathan was talking, but it came out sort of bleary, so here's my reproduction:

Things I found interesting:

Jonathan described the process Oracle Development follows for causing patches to land in infobundle.zip, the file that Patch Wizard uses to make recommendations of patches to apply to your environment. Basically, the assorted development groups tag patches that are recommended or critical patches. So you won't find every single patch that Oracle has produced to date in the infobundle.zip file, just the ones that are very important.
That said, if you discover a patch that Patch Wizard isn't telling you about, you can still use Patch Wizard to tell you more about that patch. Patch Wizard's interface is very intuitive, particularly when it comes to digging around in a patch's details to understand what the patch (or patches) will affect if applied.
If you aren't a Patch Wizard user, you can still get to the same information on My Oracle Support. Just go to the Patches & Updates tab and select Patching Quick Links. From there you can search for patches for Release 11i or Release 12. If you go that route, you'll find it a little tedious; Oracle created Patch Wizard for good reason, so it is well worth it to use their tool rather than dig through the screens on My Oracle Support.

Going back to Jonathan's slide, you can do either a Manual download or an Automatic download of the infobundle.zip data. The reason for manual downloading? Some customers don't have their production environments opened up to be able to directly download patches from Oracle. Some customers have to download the information to a separate location and then move it over. If you're in that situation, then you won't be able to make full use of Patch Wizard, which will not only download infobundle.zip - on a schedule if you want - but can also download the patches.
One caveat if you are able to automatically download - as part of the setup for Patch Wizard, you tell it somebody's My Oracle Support identification information - a valid My Oracle Support username and password. So if the password changes for that account, Patch Wizard will stop working and you'll need to figure that out and match up the password in your Patch Wizard setup with your new password. I mention this because I can almost guarantee that this will happen to me and I'll end up searching for a solution on My Oracle Support and then blush to discover that it was just a password problem.

Upstate New York Oracle Applications User Group Meeting, January 2012:

Truth be told, I haven't been to a local user group meeting in... I'm thinking 16 years. I got an invite, and realized that I've lived near Rochester, NY for 11 years and have had almost no interaction with local E-Business Suite users. If you held me up against a wall and threatened me, I'm not sure I could name five user companies. So I thought I'd give this meeting a try, and had absolutely the best time!

Doug Eberhardt, Marketing Manager at Sealing Devices, Inc. gave a terrific presentation called Choosing an e-commerce portal about his company's implementation of Workbench Software's Customer Portal for the Oracle E-Business Suite, including information on the implementation, product features, and how his company's customers have reacted to the new services. I loved this presentation for several reasons:

Doug described the process that his company went through to select Workbench Software as their e-commerce portal software vendor. A key feature of the product that Sealing Devices chose was that it interfaced seamlessly with the E-Business Suite. Patrick Harris from Workbench Software mentioned that his company had years of experience with the E-Business Suite and had used only Oracle's standard interfaces for their software.

Doug talked about how things have changed in the sales arena. I've always known that having a website was a good and useful thing, but now understand that the way that we as consumers buy things has changed a lot for many of us, and a good website can mean all the difference. I can think of several cases in the last few months where I've gone to a website, done my research, and happily made a purchase without dealing with a salesperson. Doug also made the point that if your website can answer standard questions automatically, without requiring a human to respond, you can save your company money and make customers happier there too - my favorite example is websites where I can check the status of an order or track when it will be delivered without calling in for that information.

I think, though, that what I enjoyed most about Doug's presentation was the story he told of how happy his company was with this vendor, Workbench Software. The product was easy and quick to install, maintaining it can be done by a non-technical person, and the service provided by Workbench Software sounded phenominal. Those are the kinds of stories I like to hear, where a local company provides a product and service that is so good that customers are willing to spend time putting a presentation together to talk about it.

So, folks, get out there and consider some of the many resources available for learning more about the E-Business Suite. The big kahuna of OAUG conferences is now on the horizon - Collaborate 12, April 22-26 in Las Vegas.

I'll be presenting one paper, Clean Up That Mess! A Mother's Guide to Managing Your E-Business Suite Clutter: If you’ve noticed that your Applications seem to be slowly grinding along, perhaps you need to review your overall approach to managing your Applications data. Are you purging data that ought to be purged? Is your Concurrent Manager configuration all that it could be? Have you LOOKED at your Workflow tables lately? Join me for a lively discussion of all the dust bunnies I’ve found in assorted E-Business Suite closets, and what I recommend you do to clean sweep your way to performance improvements.

Another great conference coming up February 14-16 in Denver, Colorado, is RMOUG Training Days, sponsored by the Rocky Mountain Oracle Users Group. Is it the possibility of a ski weekend that snags all the great minds for this conference? Many of my favorites, including Tim Gorman, Jonathan Lewis, Jerry Ireland, Mike Swing, Cary Milsap, and Mark Farnham will be presenting. I wish I could go, as this looks like a super, well-organized, presentation-packed conference, well worth the investment.

You Have to Be a Member, But...

2012-01-05T15:55:00.000-08:00

I write a lot. Edit a lot. Re-write a lot. My eyes are weary. Some days there is nothing more soporific than the material I am working on. But I really enjoyed working on an article that made it into OAUG's Insight Magazine, Winter 2011:

The article is called: The Big Picture - Upgrading to Release 12.1. I co-authored this with Mike Swing from TruTek. You have to be an Oracle Applications User Group member to get to the article, so if you're not and you're just dying to know what the big picture is, you can download my presentation on it. The article is prettier than my presentation, though. For a more detailed version of the topic, check out Mike Swing's paper The Big Picture - The Release 12.1.3 Upgrade Review.

Another useful link for certifications...

2012-01-05T13:12:00.000-08:00

So I published my last blog entry and then sent a note to a couple of colleagues, pointing out that I had out-blogged them, and of course my colleague James Morrow immediately responded, pointing out that this link is quite handy:

http://blogs.oracle.com/stevenChan/resource/certifications.html

James is indeed correct. I followed the link and saw something that instantly cast doubt on what Certify was noting for running Release 12.1.3 with RDBMS 10.2.0.4:

Yep, this link says that 10.2.0.4 is certified, while Certify says it is not. So, I figure this means one of the following:

Certify is wrong
There are Operating System combinations other than the one that I selected (Oracle Solaris on SPARC (64-bit) (10) that are certified with RDBMS 10.2.0.4
This link is wrong

If I were a truly dogged researcher, I would sit here and slog through My Oracle Support, searching for Operating System combinations that are certified with RDBMS 10.2.0.4. But it's the end of the day, and I'm losing my inspiration. So I'll suffice it to suggest one more pearl of wisdom - if you are researching a software combination and Certify says that it is not Certified, it would still be worthwhile to log an SR with Oracle Support to confirm it. I would think that being the people who keep Certify constantly up-to-date has to be a pretty tough job. And I am very glad it is not my job.

A Wee List of Interoperability Documents

2012-01-05T11:44:00.000-08:00

I was trolling around on the Oracle Support Community site, a dandy site because you can ask questions and your colleagues will jump in to provide guidance, when I noticed someone asking if you could use RDBMS Version 10gR2 with the E-Business Suite Release 12.1.3. I'm pretty sure the answer is yes, though I don't imagine anyone recommends you do so. My rationale stems from three sources:

1. On My Oracle Support, you can look for a certified RDBMS version under the Certify tab (I looked at 10gR2 with E-Business Suite 12.1.3 with the Sun 64 bit operating system), and there are certified versions - with caveats. One version, RDBMS 10.2.0.5 is still able to be patched if new issues are discovered, but Version 10.2.0.3 is not.
2. Also on My Oracle Support, you can look to see if there is an Interoperability document that explains how to connect your particular database version with Release 12.1.3 (you'll have to apply an Interoperability patch), and there's one of those.
3. But just because you can do something doesn't mean you should, so take a look at Oracle's Lifetime Support information and think carefully about whether you want to upgrade to Release 12.1.3 on a version of the RDBMS that Oracle may not patch if you hit an error they haven't already resolved. I'm betting that if you're considering doing it, then you must have a legitimate reason - perhaps there's a version of software, or an obscure E-Business Suite module that isn't supported on 11gR2 and thus requires RDBMS 10gR2, so maybe you're thinking you could stick with 10gR2 while your users wean themselves off of that module? That's the one example I was able to think of. The tradeoff is likely in performance and something I'll call "obscure technical issues" - those are issues that occur because the E-Business Suite Release 12.1.3 was developed using RDBMS 11g, not 10gR2, so you might find that you hit errors that nobody else hits when they're upgrading. And you may find you have technical issues with your R12.1.3 E-Business Suite environment after you've completed the upgrade that others who are running 11gR2 don't experience. I guess the bottom line is, if you were thinking you wouldn't have to test your environment too hard for this upgrade, dream on. You'll want to test extremely thoroughly if you plan on sticking with an outdated version of the RDBMS.

CERTIFY

So let's start with Certify.

Log into My Oracle Support. Click on the Certifications tab. I've chosen Oracle E-Business Suite for the Product, Release 12.1.3 for the Release, and Oracle Solaris on SPARC (64-bit) (10) for the Operating System and version.

And the answer is:

Well, good news. If this is your configuration, then yes, you could stay on RDBMS 10gR2. But notice that only 10.2.0.3 and 10.2.0.5 are certified. So if you're on 10.2.0.4, and you still wanted to stay on RDBMS 10gR2, you would need to upgrade to 10.2.0.5 to have a certified platform. If you're thinking that as long as you're in the right range of 10-ness, you should be ok, please cast that thought out. You don't want to be the guy or gal who has to explain to your management while you are in the middle of a database issue that you're having the issue because you chose not to follow Oracle's recommendations.

INTEROPERABILITY

In order to use a particular database version with a particular E-Business Suite release, you have to apply a particular interoperability patch. That patch makes them work together correctly. And it turns out that there are usually some additional patches that you need to apply - perhaps some RDBMS opatches, or some Applications patches. Those additional patches are documented in the appropriate interoperability My Oracle Support document. Here's a list of documents for several of the database version / E-Business Suite Release combinations:

If you are running Oracle 9i and need to upgrade to 10g before upgrading to RDBMS 11gR2, use MOS Doc. ID 362203.1 for the 10g – Release 11i upgrade and MOS Doc. ID: 1058763.1 for the 11g – Release 12 upgrade
If you plan to use Oracle Database 10gR2 with Release 12 (not recommended), use MOS Doc. ID 812362.1
If you plan to upgrade to RDBMS 11.1.0.7 because Release 12.1 lays down an 11.1.0.7 RDBMS (not recommended), use MOS Doc. ID: 452783.1 with Release 11i and then MOS Doc. ID: 802875.1 with Release 12.1
If you plan to upgrade the RDBMS to Version 11gR2 prior to upgrading from Release 11i to Release 12.1 (recommended), you will need to use two Interoperability documents, MOS Doc. ID: 881505.1 to deal with RDBMS patches specific to Release 11i when you do the database upgrade, and then MOS Doc. ID: 1058763.1 to deal with RDBMS patches specific to Release 12.1 when you do the applications upgrade.

These are the assorted MOS Interoperability Notes:

MOS Doc. ID: 362203.1, Oracle Applications Release 11i with Oracle 10g Release 2 (10.2.0)
MOS Doc. ID: 1058763.1, Interoperability Notes Oracle E-Business Suite Release 12 with Oracle Database 11g Release 2 (11.2.0)
MOS Doc. ID: 812362.1, Interoperability Notes Oracle EBS 12 with Oracle Database 10gR2 (10.2.0)
MOS Doc. ID: 452783.1, Interoperability Notes - Oracle E-Business Suite Release 11i with Oracle Database 11g Release 1 (11.1.0)
MOS Doc. ID: 881505.1, Interoperability Notes Oracle EBS 11i with Oracle Database 11gR2 (11.2.0)
MOS Doc. ID: 802875.1, Interoperability Notes Oracle EBS Release 12.1 with Oracle Database 11gR1 (11.1.0)

If you're wondering why I noted certain options as "not recommended", it's because I don't recommend using older versions of software that have been superseded with newer certified versions if you don't have to.

By the way, as you're working your way through the Interoperability Notes, I'd like to emphasize a truly important point: Don't skip any steps.

Some of the steps seem trivial... maybe you're thinking they don't apply to you. When we've encountered RDBMS issues during the Release 12.1.1 part of the upgrade, it's almost always because the DBA left out a step from the Interoperability document or the database upgrade document.

I'll give you one example, though this wasn't the DBA's fault - we had a client who upgraded to Release 10gR2 several years ago. The version of their documentation at the time that they upgraded didn't say anything about applying a Korean Lexers sql script that is described in the Interoperability documents (note that if you look at the documents now, they include applying this script, so this was a timing issue for the documentation).

As a result, as they upgraded to RDBMS 11.2.0.2, the DBA saw a step that said that if they were upgrading from 10gR1 or earlier, they needed to apply the Korean Lexers script. Since they were upgrading from 10gR2, they concluded they didn't need to apply the script, not realizing that the script had never been applied in their environment. The result was that the R12.1.1 part of the upgrade failed, they had to apply the Korean Lexers script, and then restart. And personally, if I were reading through the instructions, I would have seriously considered skipping the Korean Lexers script, on the assumption that it was a languages issue that didn't have anything to do with my single language environment. So, bottom line: Don't skip any steps.

LIFETIME SUPPORT INFORMATION

As you are planning this bold move, you should also take into account how long you can limp along on this version of the database. Certify has made that easy. In the previous screen, there's a list of supported RDBMS versions. Click on 10.2.0.5, and you'll see this screen:

Now click on 10.2.0.3, and you'll see the following screen:

You need to be really careful to read the fine print. Notice that for 10.2.0.5, it says "New patch creation available [CPUs, PSUs, one-offs]", while for 10.2.0.3, it says "No new patches are being created for Oracle Database 10.2.0.3.0". So what does that mean? Well, if you hit an issue with 10.2.0.3.0 that requires a patch that doesn't already exist, you're going to have to make a sudden upgrade to 10.2.0.5.0, assuming that it has that patch.

And make sure you read ALL the fine print. Click on the Oracle Solaris on SPARC (64-bit) 10 button to see more details. You may have to apply additional patches to make this version work:

So, at this point, I think we've figured out that you can run Release 12.1.3 on RDBMS 10gR2 if you are running a certified combination of the database and the operating system. If you happen to be doing so, please let me know, as I'd like to know how it all works for you.

Did You Hear That?: ATG 12.1.3 Release Update Pack Required by Feb 2012 for Premier Support

2011-12-06T16:21:00.000-08:00

I'm quoting my friend Floyd Teter, who is quoting Steven Chan from Oracle.

"ATG 12.1.3 Release Update Pack Required by Feb 2012 for Premier Support"

umm... It's December 6th. You're all getting ready for the holidays. Or your plant shutdown. Or your year end code freeze. And I just wanted to say, if you're on Release 12 and you're not on Release 12.1.3, then you need to get there. By February. And when I say February, I think that means January 31st. Not February 29th. You've probably been thinking you have until February 28th, but this year is a Leap Year (I am an expert on years in which the Leap occurs, because I only get anniversary gifts every four years). In spite of the extra day provided by Leap Year, the deadline for getting the ATG 12.1.3 Release Update Pack in is February 1, 2012. Which is, like, really soon.

To get you started down the path to R12.1.3, here are some tips:

First, be sure to read MOS Doc. ID: 1066312.1, Oracle E-Business Suite Applications Technology Readme for Release 12.1.3 (R12.ATG_PF.B.delta.3, Patch 8919491).
Make sure you're current on your RDBMS Interoperability Patches - the appropriate MOS Note will depend on which version of the RDBMS you are currently running. I see new patches for various RDBMS versions periodically, so don't skip this step. Here's the one if you're running RDBMS 11gR2: MOS Doc. ID: 1058763.1, Interoperability Notes Oracle E-Business Suite Release 12 with Oracle Database 11g Release 2 (11.2.0.2). Of course, that said, I want you to upgrade to 11.2.0.3, but we'll talk about that later.
There's been a change to the steps for applying R12.1.3, Patch 8919491 - there's a new patch, 12964564, that Oracle says you should merge with Patch 8919491. If you're in the middle of testing and didn't notice this change, MOS Doc. ID: 1066312.1 says you can tack it on afterward as a post-upgrade step if you need to.
Don't forget the Post 12.1.3 patches from MOS Doc. ID: 1066312.1.

And then, just when you think you've got all the patches you could possibly apply, you need to apply more!

If you haven't done so already, apply a patch to Patch Wizard, a free tool provided by Oracle as part of Oracle Application Manager (OAM), which is also free. But Patch Wizard needs to be patched or you'll rip your hair out trying to figure out why it isn't working. See Patch Wizard Utility [ID 976188.1], Patch Wizard FAQ [ID 976688.1], Patch:10629916 for the latest version of Patch Wizard in Oracle E-Business Suite (Release 12.0), and Patch:10629956 for the latest version of Patch Wizard in Oracle E-Business Suite (Release 12.1).
Run a query to see what modules are either shared or installed in your environment.
Configure Patch Wizard (I swear, it's really easy; you just need to check boxes for the modules you have either shared or installed in your environment). If you configure Patch Wizard, then it will tell you just the patches you need for your modules. If you don't, then it will tell you every patch for every module that Oracle offers.
Run Patch Wizard. Sort the results to make "Unapplied" patches pop to the top, Copy the unapplied patches that are listed into a spreadsheet or Word document, read all the readmes looking for additional pre-requisite patches, post-patches, or extra steps. Update your spreadsheet. Apply the patches.
Consider patching your applications server while you're at it - there's an Application Server Release 3 (10.1.3) Patchset 5 (10.1.3.5.0). Release 12.1.1 implemented 10.1.3.4. There are several Forms 10.1.2.3.0 fixes consolidated in bundle Patch 9593176.
Oh, go on, consider patching your RDBMS Version as well - Version 11.2.0.3 is now certified. And if you do that, then make sure you check for additional Interoperability patches for Version 11.2.0.3 with Release 12.1.3. And if you're going to do that, you might as well look for a higher PSU for Version 11.2.0.3.
We'll leave testing and applying the January 2012 CPU, due out in mid-January, for another day.

And BOOM, you're there. I've done this several times for an assortment of upgrades from 12.1.1 to 12.1.3, and I'm going to go out on a limb here and say that you'll probably have a lot of patches to apply (I'll say less than 200, but Release 12.1.3 counts as a pretty darned big patch), but they probably won't cause you issues in applying them. So, applying the patches will probably not be a bad experience, so be kind to your test team and get that part out of the way ASAP so they can have as much time as possible to test what you've given them.

You can download a free copy of Red River's Oracle E-Business Suite Release 11i & Release 12 Patching 101, which steps you through how to set up and use Patch Wizard. I have a vested interest in this book, as I wrote it with John Stouffer, and Mike Swing from TruTek provided the test environment for pounding through Patch Wizard issues. We'll release a new issue right before Collaborate 2012 to update the patch list and other modifications that Oracle has made since last year.

Shared HR Customers: Now That You've Made it to R12.1.3, Should You Upgrade to HR RUP4?

2011-12-06T14:04:00.000-08:00

I am working right now with an E-Business Suite client who is a shared HR/Payroll user. My client is testing the upgrade from Release 11.5.10.2 to Release 12.1.3+. If you've gone through the upgrade, you may sympathize when I say that every time I tell the client that there's one more patch to apply, I can actually hear his eyes rolling around in his head.

So, when you upgrade to Release 12.1.1, you upgrade to HR RUP3. And if you run patchsets.sh afterward, you'll notice that HR RUP4 is now available. And if you patch Patch Wizard and run Patch Wizard, it will tell you about some one-off patches for HR, as well as recommend that you apply HR RUP4. So here's the question: must you?

I did a little research. First, I have to tell you that I've become a My Oracle Support Known Issues Note Junkie. Oracle creates Known Issues documents to correspond to big patches that they've released. The Known Issues notes track outstanding issues and patches for those issues if they are available. And yes, there's a Known Issues Note for HR RUP3. So if you've upgraded to Release 12.1.3, you're already behind and need to take a look at the Known Issues Note. There's also a Known Issues Note for HR RUP4.

I trudged through those two Known Issues notes, and found that for HR RUP3, there are 55 additional recommended patches. That's as of 11/24/2011, the last time I looked. There may be more by now. And for HR RUP4, there were 24 additional recommended patches. Hmmm... 55 patches... 24 patches. I weeded through the Readmes for both MOS notes, tracked any additional pre-requisites, built myself a list for HR RUP3 and HR RUP4, waffled several times over what to do, and finally made this recommendation to my client:

Upgrade to HR RUP4, and apply the Known Issues patches for HR RUP4 that apply to your environment.

We weeded out, for example, patches that were legislative related. Shared users don't apply legislative patches. We weeded out patches for modules that weren't installed. By the time we were done, for this client, we were able to cut down to the HR RUP4 patch plus six additional patches from the HR RUP4 Known Issues document.

My client applied the patches and they all went in smoothly. We haven't tested yet, as he is sorting through the Patch Wizard-recommended patches today, but still, what we did apply applied smoothly. I can't guarantee that you'll be so lucky, since your data is different and you may have different shared and fully installed modules, but I did find it comforting to be able to say that for this client, applying HR RUP4 did not cause any issues during the application process.

Back to the original question: Must you upgrade to HR RUP4? Could you apply the HR RUP3 patches and let it go at that? Could you skip the extra HR RUP3 patches?

I think the answer to this kind of question always falls back to how risk averse you are. And maybe how solid your test team is. Oh, and it really matters if you're a fully installed HR/Payroll user or a shared installed HR/Payroll user.

If you have a super solid test team and they've thoroughly tested your R12.1.3 instance and they aren't seeing any issues, and if you've passed the HR RUP3 Known Issues patch list by them, and they don't believe that any of those patches are necessary, and if you're an HR/Payroll shared mode user, then you could probably stop where you are. Probably. Me, I'm so freaking risk averse that I would feel compelled to apply more patches. My concern is always that a change in a patch that is described in a readme will affect something that isn't described in the software that I care about.

On the other hand, my understanding is that if you are a fully installed HR/Payroll user, then you're supposed to stay current on patches, so you need to apply HR RUP4, and you need to follow the instructions that describe running hrglobal.drv for legislations, and you need to find the HR RUP4 Known Issues document and apply those patches, and you need to continue to monitor the HR RUP4 Known Issues document for more patches.

That said, I'm very interested in HR right now, so if you have any opinions, I'd love to hear from you.

psst... I found a MOS Note that describes how to switch from Fully Installed to Shared for HR

2011-12-06T13:16:00.000-08:00

A couple of weeks ago I was trolling around on My Oracle Support, something I do quite frequently to stay current on the latest patches for Release 12.1.3, and I found a super-interesting MOS Note:

Changing Oracle Human Resources Installation from FULL HR/FULL Payroll Install to SHARED HR/SHARED Payroll Install [ID 461063.1]

Now that's exciting. Aren't you excited? If you're not familiar with HR, the issue here is that traditionally, Oracle has said that they wouldn't support changing the status of your HR module from fully installed to shared install. So seeing instructions on how to do so, and the caveats to consider, is just plain interesting.

The note describes two supported ways of making the change:

1. If you've never run the hrglobal.drv program against your instance, then it is very easy to switch from fully installed to shared installed. Customers in this situation likely inadvertently chose to fully install HR/PAYROLL when installing the E-Business Suite in the first place, but really had no intention of using the additional functionality or licensing those modules. Those customers correctly ignored instructions in various patches to run hrglobal.drv over the years, because they didn't need to run it. So, if you really intended to be a shared user, and you never ran hrglobal.drv, then MOS Note 461063.1 includes a script that will flip the switch on the STATUS column in the FND_PRODUCT_INSTALLATIONS table. MOS Note 461063.1 also includes a script you can run to see if you've run hrglobal.drv, so it's very easy to figure out if you're in this situation. The MOS note also includes a script to change the status of some other HR/Payroll-related modules.

Now, I couldn't resist checking at one of my clients who is already in shared mode, so they ran the query for me, and it turns out that they've run hrglobal.drv twice, once in 2006 and again in 2011. I mention this because to me this suggests that folks are confused about what they're supposed to be doing when it comes to HR. I looked over various MOS notes, and have to admit that I was confused. You have to understand what Oracle is talking about when they discuss legislations, and if you don't, then it would be easy to conclude that applying something you might not need is easier than not applying something you might need. Personally, I think it would be nice if any patch that includes hrglobal.drv also checks to see if HR/Payroll is fully installed, but I am not an Oracle programmer so the solution may be more involved than my simple perspective surmises.

Anyway, the MOS document includes some more queries and changes that you need to make to fully apply these changes, so check it out if you're interested.

2. The more difficult scenario occurs if you have HR fully installed, you wish you hadn't, and you have run hrglobal.drv. I'm copying straight from the MOS note to avoid any misinterpretation here:

To be SUPPORTED as a SHARED HR Installed instance you will have to do the following:

3a. Create a 'brand new instance' installed as SHARED HR Install (also you will have to verify the STATUS for the other HR related products and change their STATUS appropriately (Oracle provides a list in the MOS Note))

3b. You will then need to export your table data from your FULL HR Install instance to your newly created SHARED HR Install instance where the hrglobal.drv script has 'never been run'. This means 'all of your table data for all Oracle applications'.

3c. This is the process for changing Oracle Human Resources Installation from FULL HR Install to SHARED HR Install where the hrglobal.drv script has been run (legislations ARE installed on your instance).

I have to admit, I'm utterly intrigued by this option, but concede I would be worried that I might accidentally include table data related to being fully installed in the import from the original Full HR instance described in Step 3b. But I'm betting that logging an SR with Oracle could result in enough information to do the deed. You can see that making this change is a significant undertaking, and your test team would definitely have to put your instance through a thorough test process. But it's nice to see that there is a way to get there if you want to.

With all that said, if you've followed this process, or come up with your own process for making this change, I would love to hear more details about it and whether you ran into any additional issues. Also, how did your test team test that the conversion worked? Do tell me. I'm all ears.

The Subtleties of CPU Application Between Releases

2011-12-06T11:49:00.000-08:00

I've been plugging away on an R11.5.10 to R12.1.3 upgrade over the last couple of weeks, and ran into an issue. My client made it all the way in a test upgrade to Release 12.1.3 and then created a merged patch of some additional recommended patches, including the October 2011 CPU. And, well, it failed. Here's the message he saw:

afmimefilext.sql

sql /yyy/a/oraapp/xxx/apps/apps_st/appl/fnd/12.0.0/patch/115/sql/afmimefilext.sql &un_fnd &pw_fnd

Start time for statement below is: Thu Dec 01 2011 15:36:49

INSERT INTO FND_MIME_TYPES ( mime_type_id, mime_type, ctx_format_code,
last_update_date, last_updated_by,creation_date, created_by,
last_update_login,file_ext, allow_file_upload ) VALUES (
fnd_mime_types_s.NEXTVAL, 'application/zip', 'IGNORE', sysdate, -1,
sysdate,-1, -1, 'zip', 'N')

AD Worker error:

The following ORACLE error:

ORA-00001: unique constraint (APPLSYS.FND_MIME_TYPES_U2) violated

With research, we concluded that the problem was occuring with the Oct 2011 CPU for Release 12, which includes two patches, 12794417 and 12921332. With more research, we realized that as part of the RUP7 "Known Issues" document, we had already applied the Oct 2011 CPU, but on Release 11i, rather than on Release 12. And with more research, we realized that Oracle gives the Release 11i CPU patch a different patch number than the Release 12 CPU.

Hmmph. I did not know that. I had been counting on adpatch to screen out duplicate patches between Release 11i and Release 12, so it hadn't occured to me to worry about whether the latest CPU had already been applied.

Our solution? For this pass of the upgrade (a test pass), we rolled back to before the post-Release 12.1.3 merged patch was applied, but after the Release 11i Oct 2011 CPU had been applied (because that occurred at the very beginning of the upgrade), took the Release 12 Oct 2011 CPU out of our merged patch, and then applied the merge again, with no problems.

Considerations? Well, for the next pass, we'll remove the Release 11i Oct 2011 CPU from our collection of patches that we apply to Release 11i before upgrading. We'll add the Release 12 Oct 2011 CPU back in to the merge that we ran after upgrading to Release 12.1.3. And for future upgrades, we'll make sure we're tracking which CPU has been applied on which E-Business Suite release more carefully, because it sure doesn't work to apply the CPU twice.

Six Presentations Ready for Connection Point EPM/BI/R12.1 Next Week

2011-11-11T08:32:00.000-08:00

There's nothing like preparing for a conference to force getting organized. I've been working on presentations for OAUG's upcoming Connection Point EPM/BI/R12.1 Conference, to be held on November 15-16 in Atlanta, Georgia. If you can't go, you might still want to check out the presentations, as several look very interesting. Presentations will be online next week at the OAUG website. Just look under the Education & Events tab and click on Conference Paper Database.

Here are the ones I've been working on:

With Donna Campbell at TruTek:

What You Need to Know About Identifying Customizations Before Upgrading to Release 12.1

Are you ready to upgrade, part with, or overhaul your customizations for Release 12.1? This presentation will give you the skinny on what you need to know, including how to identify your customizations, what types of changes constitute a customization, how to determine if you should keep or eliminate a customization, and what to worry about for those customizations you decide to maintain.

We've overhauled this presentation considerably since we first gave it at Connection Point 12.1 in July. The most fun part was when we added in an example of a customization, one that I created 15 years ago. While I doubt that it is still in place, I was trolling through the list of attendees for Connection Point and noticed two people from the company that I worked for at the time will be at the conference. I guess I'll have to check to see if my sad looking code is still there.

With John Stouffer at Just a DBA:

E-Business Suite Release 12 Apps DBA 101

Learn about the new features, functionality and utilities available to Applications DBAs with R12. We will compare the R11i and R12 architectures and then describe tools including Oracle Enterprise Manager (OEM), Oracle Applications Manager (OAM), the E-Business Suite Plug-in (formerly Applications Management (AMP) and Change Management Packs (ACP)), Rapid Install Wizard, Rapid Clone, AutoConfig, Patch Wizard, OPatch and napply CPU, Oracle EBS Diagnostics, and the Oracle Integration Repository (iRep). This paper also discusses considerations for whether and when you should consider upgrading to EBS Release 12 and RDBMS Version 11gR2.

To me, this presentation is the starting point for technical folks working with the E-Business Suite. There are so many terms and products and module names bandied about, and keeping them straight is difficult. John has a companion book, Oracle E-Business Suite Release 11i & Release 12 Apps DBA 101. You can download it for free from Red River Solutions, or pick up a copy from them at Connection Point or Collaborate.

Best of Oracle OpenWorld: What You Need to Know About E-Business Application Release 12.2

While it’s still too early to make the jump to Release 12.2 (it hasn’t been released yet!), Oracle has plenty to say about what this new release will offer. Release 12.2 will replace several of the technology components with WebLogic components. Attend this session to hear about new features, what organizational impacts to consider while waiting for this release, and how to prepare for the new WebLogic software and Release 12.2!

If you didn't make it to OpenWorld this year, there were several really interesting presentations. We wanted to be sure to cover the important ones at Connection Point, so John will discuss Release 12.2, and there will also be a panel covering a few of the more important presentations regarding the E-Business Suite:

Kaberi Nayak, Mitre: Prasad Akkiraju and Steven Chan’s Session 17249, Oracle E-Business Suite Technology Certification Primer and Roadmap
Alyssa Johnson, Solution Beacon: Cliff Godwin’s Session 16800, General Session: Oracle E-Business Suite: Vision, Strategy, and Roadmap
Bill Dunham, OATC: Sara Woodhull’s Session 17250, Upgrading Your Customizations to Oracle E-Business Suite 12.1
Mike Swing, TruTek: Nadia Bendjedou’s Session 16541, Coexistence of Oracle E-Business Suite and Fusion Applications: Technical Dive

The Patch Wizard Path to Fewer Errors

The quantity of patches available for EBS customers can seem overwhelming. There are CPUs/PSUs, RUPs, Family Packs, Mini-Packs and one-off patches. Oracle provides patchset.sh to show customers their high level patch situation, including family packs and mini-packs. B ut there are still more patches available that customers may not see if they don’t dig deep enough. The EBS Patch Wizard helps customers decide which patches to apply and what data the patch will change. It compares the list of applied patches to available patches (by downloading the Patch Information Bundle) and makes recommendations. Learn how to take all this information and plan out which patches to test and apply next, with examples for both R11i and R12.

I heart Patch Wizard. Really. I use it all the time, and I talk about it as if I invented it. John has a companion book for this presentation, Oracle E-Business Suite Release 11i & Release 12 Patching 101. You can download it from Red River Solutions, or pick up a copy from them at Connection Point or Collaborate.

All by my lonesome:

Clean Up That Mess! A Mother’s Guide to Managing Your E-Business Suite Clutter

If you’ve noticed that your Applications seem to be slowly grinding along, perhaps you need to review your overall approach to managing your Applications data. Are you purging data that ought to be purged? Is your Concurrent Manager configuration all that it could be? Have you looked at your Workflow tables lately? Join me for a lively discussion of all the dust bunnies I’ve found in assorted E-Business Suite closets, and what I recommend you do to clean sweep your way to performance improvements.

Lessons from the Front: Upgrading to Release 12.1

We’ve spent the last nine months working on Release 12.1.3 Upgrade Readiness Assessments and First Pass Upgrades. The Assessments help determine if there are key steps — like making changes to your hardware and software — that ought to be completed before you upgrade to Release 12.1.3. The First Pass Upgrades are just what you’d expect — the first try, where you take your Release 11i environment, clone it, and then follow the many steps required to upgrade your environment to Release 12.1.3. This presentation will describe what we’ve learned from repeated upgrades with several different clients. The most important lesson we’ve learned? Anything you can do in advance to “pre-fix” your data will save you hours of time during your First Pass.

This presentation is based on an article that Mike Swing (TruTek) and I wrote for OAUG's winter Insight Magazine, coming out in December.

Why It Was Not So Terribly Bad to Have Missed OOW This Year...

2011-10-30T13:21:00.000-07:00

Sacrilege! Did I just name my article the equivalent of "Join Me in Skipping OOW, You Troublemaking Scamps"?

I am very sorry. This year I was totally prepared to go. I had my plane ticket and a free pass. I was totally beefed up to ride the BART, even from the AIRPORT, a not insignicant willingness on my part because of a long ago acquired fear of public transportation. But my Dad wasn't doing well. So I cancelled and brought him his last McDonalds hot fudge sundae, and then he was gone. As it turns out, my brother and sister-in-law managed to food poison themselves on the Wednesday that I would have been returning from the conference (I was planning to stay with them), so I even managed to avoid that agony. So we here at OnCallDBA are thoroughly convinced that sacrificing the conference was the right way to go (this time, anyway).

With that said, though, I don't want you to think that the conference didn't go on without me. And there were some swell sessions, ones that were so good that they bear remarking upon. In fact, my reading of the presentation slides was so exciting that I did something I swore I would never do - I actually recommended a PANEL for another conference. Oh yes, get to know me and you'll find there are at least 83 things that I have, over the course of my life, concluded I do not like or want to recommend to anyone else. Panels hit the list one ill-fated year when I was asked to be on a panel at Collaborate. This was quite a while back, back when they had those big microphones that needed to be passed from person to person. There was this one fellow, whom I'll call "Mr. Microphone Hogger", and then I was next to him, and then there was this other fellow, whom we might just as well call "Not Close Enough to the Microphone". Mr. Microphone Hogger held onto that microphone and dazzled the audience with responses to their questions for an hour. Our Moderator apparently concluded that introducing us by name was the most important job he could do, and went mute after the introductions. Occasionally, I managed to pry Mr. Microphone Hogger's fingers off of the microphone long enough so that I could toss in a comment or two by bending the microphone toward me. And poor "Mr. Not Close Enough", well, he would have had to lean in front of me to snag that microphone, so he didn't get to say word one. And that was it, that was the year I concluded that panels were not for me.

So now you're wondering, "What panel could this panelphobic non-OOW-attending person possibly recommend?" Well, I've been working on the OAUG Connection Point 12.1 conferences, and I concluded that we needed a panel called: Best of OOW Panel - Four Presentations We Just Can't Stop Talking About. Oh, but it wasn't enough for me to throw a bunch of panelists in front of an audience and get them talking. Nope, the school marm in me came out (although I am not a school marm in real life, I am certainly an overbearing taskmaster), and I picked four people who attended OOW and had good things to say about the presentations, and I asked them to pick their favorite Oracle presentation, and then, just to make sure there was some real skin in the game, I asked them each to write an article about what they learned. Phew! The panelists are exhausted. They struggled to decide which presentation was their absolute favorite. They poured over the presentation slides, which, by the way, are available to EV-ERY-ONE this year, instead of just to people who attended OOW, and then they wrote. And wrote. I was actually a little stunned by how much effort my panelists put into their work. I was thinking maybe a couple of pages, but my treasured panelists got down to work and wrote thoughtfully and thoroughly about their topics.

So, with that said, the next tough job was putting together a newsletter that includes the articles. Here's a link: http://campaign.r20.constantcontact.com/render?llr=574eipcab&v=001bNaaDsMZRRdPhsclrMifR34aOZbF1jRdkqf7FF0c94_PLv8bau-OCevkzCCNaT-6kSoa6IFwWhTnuIFz6pLcdC3mhBECLXwTXzSpeW_2lm7DrUWLaDiI4MqXI8Plnj0veJ_MMHGJ7dtkjTga49iJJVVciqjr_uE4B-TK8vD3aU-Ad1JoQ5A0EMsiQ-0qMRgEn3odD5ua453P4z2S3Xb9kMA453iFo1--39BU5UGO2Z4i9uINe52R5H06lyA1ehflQNCJ7la_J6pKvsI8aJY2JryU6FircKnHlU0oDc_MaqAzTDjE2VI4Kxu1G_qrK9Hm_1b7TCpnYm6s3zvK5zhDaGvAkVJps301iPZa0lcE6ochIlQlO7eC2aj636aG4Qa0

In the meantime, the conference in question is OAUG Connection Point - EPM/BI/R12.1 Upgrade. It is coming up quick - November 15-16 in Atlanta, Georgia. And the agenda is AMAZING. If you're thinking about upgrading to Release 12.1, then you need to consider attending this conference. And if you can't make it to this one, the OAUG is working on another one in February in San Diego. Dates haven't been announced yet for that one. I had never attended these OAUG Connection Point conferences until the July 2011 Connection Point Release 12.1 Conference in Chicago, but they offer a terrific way to learn what you need to learn. The crowd is totally focused on the topics at hand, the speakers are some of the best in the business, and the conference is relatively short - 2 days - so you don't have to lose a whole week of work.

I forgot to tell you what the four presentations are, didn't I? Sorry about that. Here's the list:

Kaberi Nayak, Mitre: Prasad Akkiraju and Steven Chan’s Session 17249, Oracle E-Business Suite Technology Certification Primer and Roadmap

Alyssa Johnson, Solution Beacon: Cliff Godwin’s Session 16800, General Session: Oracle E-Business Suite - Vision, Strategy, and Roadmap

Bill Dunham, OATC: Sara Woodhull’s Session 17250, Upgrading Your Customizations to Oracle E-Business Suite 12.1

Mike Swing, TruTek: Nadia Bendjedou’s Session 16541, Coexistence of Oracle E-Business Suite and Fusion Applications: Technical Dive

If you decide to download presentations from OOW, just go to the OpenWorld Content Catalog. Once you find a presentation of interest, click on the pdf icon to its right.

Also, the OAUG plans to have all the Connection Point presentations available by conference time on the OAUG website, so you'll be able to find those as well.

OATM and the Release 12 Upgrade: OATM Caveats - Part III

2011-10-30T10:23:00.000-07:00

I mentioned in another post that a client had run into difficulty migrating to OATM. So I figure it behooves me to mention that there are some caveats to bear in mind when you are planning your migration. My colleague Mike Swing from TruTek dug out some excellent points from assorted MOS notes.

1. TIMING: Oracle's Release 12.1.1 Upgrade Guide actually says: "We strongly recommend that you migrate the existing objects after the upgrade is complete. Use the Tablespace Migration Utility (introduced in Release 11i) to perform this task."

What? I thought the leaning was to migrate to OATM before the R12.1 upgrade, not after! My conclusion, after much nattering, is that there are pros and cons to either approach. Oracle was probably trying to make sure you actually do the migration, rather than provide a blanket recommendation that you do so after the upgrade. In fact, Mike Swing from TruTek dug out another note that said "We strongly recommend that you migrate the existing objects after the upgrade is complete. The reason for that is because the new products created with the R12 upgrade are created in the OATM model, but the old objects are still kept in the old tablespace model. This would keep the DB temporarily with a hybrid model, what is not recommended. Probably the best approach to avoid a longer downtime during the R12 upgrade is to migrate to OATM model before that." So I think we can reasonably conclude that no matter when you do the migration, the key is to do it and don't linger with a hybrid tablespace model.

I did ask Oracle, though I haven't gotten around to asking Oracle Support for an opinion, about OATM migration timing. Max Arderius, who is Oracle's Development Manager for the Applications Technology Group, said "There is no exact recommendation for this. It really depends on each environment and the needs for each customer. All I can say is that most customers choose to migrate to OATM before the upgrade so they can minimize the downtime window. Otherwise there is still another task that will need to be completed after the upgrade." Max makes a good point. You've just finished doing a huuuuge complicated upgrade, and before your DBA can run off to celebrate at Disneyland, another project with another downtime window will need to be done if you haven't already knocked the OATM migration off your list in advance.

2. MANDATORY-NESS: Is OATM Mandatory for Release 12?

MOS Doc. ID: 418225.1, "FAQ - OATM - Oracle Applications Tablespace Migration" says: Yes. OATM is mandatory for R12, although it is not mandatory to upgrade to OATM at the same time as the R12 upgrade, as stated at:

http://download.oracle.com/docs/cd/B40089_08/current/acrobat/r12upg11i.pdf (page 1-8)

2. TOOL FABULOSITY: OATM is neither psychic (please picture me saying that in my high-pitched voice, with just a bit of added emphasis on the first syllable) nor all encompassing.

The following schemas will not be moved by OATM:

MDSYS , OUTLN, SCOTT, ORDSYS, SYSTEM, EUL4_US, SLADMIN, PERFSTAT, PORTAL30, PORTAL30_SSO, PORTAL30_DEMO
Custom schemas not registered with the Applications
CTXSYS, if not enabled for the migration. If you want to move the CTXSYS schema, please refer to MOS Doc. ID: 417742.1, OATM MIGRATION OF CTXSYS OBJECTS FAIL
There may be issues with CTXSYS, JTF and ABM schemas that require special fixes.

I mention this so that you'll be sure to check after you migrate to OATM to make sure that everything you expected to migrate actually has migrated. You may need to do additional research on Oracle Support to figure out how to move the rest.

NOTE: If you've upgraded to OATM after upgrading to Release 12, please satisfy my curiosity and let me know if there were any OATM-enabled patches for your existing modules that had to be changed to point to your existing tablespaces during the upgrade.

3. FASCINATING READING: MOS Notes About OATM For Your Review:

Oracle E-Business Suite Upgrade Guide Release 11i to 12.1.3 Part No. E16342-03, August 2010
Oracle Applications System Administrator's Guide - Configuration
Oracle Applications Concepts
Patch 3381489 - the Oracle Applications Tablespace Migration Utility patch
Oracle Applications Tablespace Model Release 11i - Tablespace Migration Utility [ID 248857.1]
How to run OATM migration utility [ID 404954.1]
OATM errors out when moving Intermedia indexes [ID 365466.1]
OATM migration leaves WF_CONTROL objects in the old tablespace [ID 418238.1]
Unable to drop tablespace APPLSYSD when migrating to OATM [ID 369526.1]
How to move objects left in the old tablespace after OATM migration? [ID 463271.1]
FAQ - OATM - Oracle Applications Tablespace Migration [ID 418225.1]
R12 : After OATM Migration, Some objects left in JTFD TABLESPACE [ID 839099.1]

OATM and the Release 12 Upgrade: When Should You Migrate? - Part II

2011-10-30T09:30:00.000-07:00

In my last post, I concluded that Oracle E-Business Suite customers really must, at some point, migrate to OATM. Now the question is, when is the best time to do the migration?

Let's start with Mike Swing from TruTek's diagram of the Release 12 Upgrade:

You can see from the diagram that you can upgrade before or after.

The Case for Migrating BEFORE Upgrading to Release 12:

I'm a big advocate of cleaning up your environment before you upgrade. Purge what can be purged and deal with fragmentation before the upgrade, on the theory that the upgrade downtime window will be less on a consolidated environment.

I also believe that any upgrade steps that can be done before the upgrade should be considered, again, to lesson the downtime window. An additional benefit - the less work you need to do during your critical downtime window, the less likely you are to experience problems. No one wants to get partway through the production upgrade and then have to cancel it because of technical issues.

The potential tradeoff, of course, is whether your data after the upgrade ends up needing another round of cleanup. Another colleague, John Stouffer, an independent consultant that I work with on E-Business Suite Assessments, says that it is worth considering that side of the coin as well.

The Case for Migrating AFTER Upgrading to Release 12:

Your database has just been trounced upon by hundreds of thousands of programs. To what effect? I'll update this post when I've heard back from John Stouffer, as I'm hoping he has some insights. In the meantime, I'll also run queries against clients I've worked with who implemented OATM before the upgrade to R12. Those all included Assessments, so I should have the data on how fragmented they were before they started the upgrade, and whether their data is just as fragmented, more fragmented, or less fragmented afterward.

Practical Experience:

Mike Swing pointed out that for his Release 11i to Release 12 training classes, he always uses the Release 11i Vision instance provided by Oracle that is already migrated to OATM. He has never had a customer not migrate ahead of time. The migration is not a small task; it takes time, so if you tried to do it during your upgrade, it could add a significant amount of time to your upgrade process.
TruTek has one customer who migrated during their Release 12 upgrade because the original instance was running Release 11.5.7; migrating to OATM would have required upgrading first to Release 11.5.10. Ultimately, this client had to do their entire upgrade in one stretch, so they upgraded the database from RDBMS 9i to 10g, the applications from Release 11.5.7 to Release 11.5.10.2, migrated to OATM, then upgraded to Release 12.1.3+ with RDBMS 11gR2. The downtime for this upgrade for this customer with a relatively small data footprint, since they only have a few modules fully installed, ended up taking 6 days, with the OATM upgrade taking 6 hours. Lest you panic about the overall length of this upgrade, keep in mind that whether there's a lot of data or not, this client had to upgrade both their database and applications twice. While they might have shaved off some time by throwing hardware at the problem, with this much change, there's only so much you can do to speed up the overall timeframe.
Another of TruTek's clients told us that he migrated to OATM way in advance, when his company upgraded their Release 11i instance to RDBMS 10gR2. His migration did not go well - his test upgrades went flawlessly, but when he ran the migration for production, he encountered a number of issues. He commented that for his environment, the physical space saving was "pretty incredible", and offered to take a look after one of his R12 upgrade test runs to see how fragmented the database is after the upgrade.

OATM and the Release 12 Upgrade: Must You Migrate? - Part I

2011-10-30T06:47:00.000-07:00

Last night I couldn't fall asleep. So I thumped on my husband's arm and insisted on telling him everything I learned this week about OATM. The man is a saint - he clearly understands the virtue of getting something off your chest, and so patiently listened to what I had to say. Of course, the remarkable thing about my husband is that he does not use Oracle, not in any way, so allowing me to interrupt his sleep with a discourse about OATM was truly above and beyond the call of duty.

The topic is still on my mind, though, so I'll cover it here, which seems like a more appropriate venue.

Let's start with the terminology: OATM is the Oracle Applications Tablespace Model. It's a framework for storing your data for the Oracle E-Business Suite of Applications. Oracle would like you to use their tablespace layout, and they've created a tool to help you migrate your data into those tablespaces. Back in the early days of the E-Business Suite, back before there were more than 200 unique modules, we kept our data in separate tablespaces, two for each module - one for the data and one for the indexes. There was something to be said for this model because back then we had small disks and plenty of them, so we DBAs enjoyed separating our data across as many spindles as possible. As time passed, Oracle introduced more and more modules, and so now managing more than 400 tablespaces isn't practical. Enter OATM.

As you plan your upgrade to Release 12.1... or Fusion... or Release 12.2, you should certainly be thinking about housekeeping. Yep, there's nothing like a full-on upgrade to make housekeeping rise to the top as something that ought to be done. As you prepare to upgrade, or even simply ponder upgrading, it's worthwhile to consider when you should migrate to OATM. I suppose you could also consider whether to migrate to OATM, but the pressure is on and Oracle expects you to get there at some point, at the very latest by the end of your upgrade to Release 12.1.3. If you decided not to do the migration, then I suppose the risk would be that there may be patches out there that assume you've migrated, so they'll fail when you try to apply them, searching for a tablespace that doesn't exist. This isn't the end of the world, but it'll stop you in your work temporarily while you create new tablespaces. Oracle does note that when they implement new modules during the upgrade, they'll place objects in OATM tablespaces. So when all is said and done, if you don't migrate to OATM before you upgrade to Release 12, you'll have a hybrid tablespace configuration with your existing tablespaces broken out by module, and twelve locally managed tablespaces for all products, including a temporary tablespace, a system tablespace, and undo segments until you migrate to OATM.

Now, there are virtues to migrating to OATM. According to Oracle, OATM:

Uses fewer and more consolidated tablespaces
Uses Locally Managed Tablespaces
Accounts for the I/O characteristics of an object
Reclaims space after migration
Provides Real Application Cluster (RAC) Support

So I'm convinced. The move needs to be made. But what got me to stewing into the late hours is the question of timing. When is the best time to migrate to OATM - before the upgrade to R12, during the upgrade, or after the upgrade?

Let's look at a diagram from Mike Swing at TruTek:

I work with Mike on upgrades, so I reference this diagram constantly. You'll notice, he gives you the opportunity to convert to OATM twice - either before the upgrade or afterward. And, technically, if you do the conversion before the upgrade, the timing could occur as part of your upgrade. That got me to thinking: when is the best time to migrate to OATM? Is there any advantage to upgrading afterward? How about doing one big migration, complete with OATM, during the Release 12.1.3 upgrade? Are there ever cases when you might choose to do that? And finally, are there advantages to upgrading in advance?

I'll cover this topic in my next blog entry.

Gobs of LOBS

2010-11-09T09:56:00.000-08:00

All right, I’ll admit it. LOBS have been bothering me since the first time I saw them in the E-Business Suite. They’re big. Really big. And I’ve never been able to find out enough information about them. So every time I look at a client’s data, there they are, lurking, taking up all that space. But this month, I decided to poke around a little on My Oracle Support. Call it my very personal quest for information about LOBS, if you will. And the good news is, sometime within the last year or so, Oracle has provided a lot more information about LOBS. They’ve even provided scripts that provide some perspective on what’s going on with those LOBS. And they’ve even provided some suggestions for how to tame your LOBS.

FND_LOBS

MOS Doc. ID: 298698.1, "Avoiding abnormal growth of FND_LOBS table in Applications 11i" tells us, "Of all the tables that occupy a very large amount of space within the APPLSYSD and APPLSYSX tablespaces of Oracle Applications instances, FND_LOBS is usually one of the top 10. This is because it stores all the attachments that have been uploaded to Oracle Applications. There is a LOB field within this table called FILE_DATA, and the corresponding LOB segment (e.g., APPLSYS.SYS_IL0000680397C00004$$) is where the actual attachment data is stored, and it is usually very large. It is important that its size be controlled."

MOS Doc. ID: 829235.1, "FAQ - Performance considerations for FND_LOBS" describes LOBS further: "FND_LOBS stores information about all LOBs managed by the Generic File Manager (GFM). Each row includes the file identifier, name, content-type, and actual data. Each row also includes the dates the file was uploaded and when it will expire, the associated program name and tag, and the language and Oracle characterset. The file data, which is a binary LOB, is stored exactly as it is uploaded from a client browser, which means that no translation work is required during a download to make it HTTP compliant. Therefore uploads from non-browser sources will have to prepare the contents appropriately (for instance, separating lines with CRLF). The program_name and program_tag may be used by clients of the GFM for any purpose, such as striping, partitioning, or purging the table if the program is de-installed. They are otherwise strictly informative.

Some of the data that gets into this table belongs to old/expired exports. For every request for an export, an entry in the FND_LOBS table is recorded. This data must be purged regularly. There is a purge program available to purge this data called "Purge Obsolete Generic File Manager Data" (FNDGFMPR). FNDGFMPR is a PL/SQL procedure that deletes old obsolete uploaded files (loaded to the database) for the programs FND_HELP, export and FND_ATTACH, these are programs that are run under the FNDGFU (Generic File Manager Access Utility). MOS Doc. ID: 216541.1 describes how to add The Concurrent Program "Purge Obsolete Generic File Manager Data" To The Sysadmin User."

MOS Doc. ID: 1165208.1, "Questions on Purge Obsolete Generic File Manager Data" says that the tables affected by the Purge Obsolete Generic File Manager Data Concurrent Program are FND_LOBS and FND_LOB_ACCESS. And don't think that I'm the only one fussing about the size of that table: there are at least two enhancements requests asking for ways to more effectively purge the FND_LOBS table. For the PO module, for example, there are two enhancement requests for deleting attachments:

Bug 3899857 MASS DELETE PROGRAM FOR DELETING ATTACHMENT - Purchasing does not set an expiration date on the attachments. Also, there is no concurrent request available currently that deletes Purchasing attachments in bulk. Enhancement Request Bug 3899857 is already logged with Oracle Development requesting this functionality.
Bug 5676144 NEED A WAY TO CONTROL THE GROWTH OF PO ATTACHMENTS IN THE FND_LOBS TABLE - requests a functionality to allow purging PDF attachments created by the PO Output for Communication for approved purchase orders. Usually the rapid increase in table space is caused by large PDF files, generated because of using large sized files for the company logos (header or footer). As a workaround, one can consider using smaller sized files for the logos.

Progress is being made - the Order Entry module already has Concurrent Programs that purge from the FND_LOBS table:

Concurrent Program: OEXPURGS - Order Purge Selection
Concurrent Program: OEXPURGE - Order Purge

If you run on the conservative side and don't want to take a chance on inadvertently deleting attachments, the Purge Obsolete Generic File Manager Data program will let you choose what types of files you purge. In this example, we limited the file types to Exports:

We ran the queries included in the assorted MOS Documents and the "Purge Obsolete Generic File Manager Data" Concurrent Program against a client's database to see what we could conclude about their LOBS:

PURGE-ABLE DATA

The Concurrent Program "Purge Obsolete Generic File Manager Data" will not purge entries for the Application Help (iHelp) and will only purge attachments or exports if they are expired. Oracle notes that the expiration of attachments should be done via an application, and not manually updating the table, so don't try to clean up the FND_LOBS table by using TOAD.

You can see entries that have an expiration date by program_name with the following query:

select program_name,count(*)
from FND_LOBS
where expiration_date is not NULL
group by program_name;

PROGRAM_NAME COUNT(*)

-------------- ----------
export 7694 - related to Export, these will be purged

Recommendation: Schedule the “Purge Obsolete Generic File Manager Data” to run periodically. Then re-run the query above to make sure it has successfully removed the expired data.

Our client ran the purge and it deleted all of the exports and freed up about 125M of space within the FND_LOBS table.

DATA THAT WILL NOT BE PURGED

Entries with no expiration date, which means they will not be purged, can be found by running this query:

select program_name,count(*)

from FND_LOBS
where expiration_date is NULL
group by program_name;

PROGRAM_NAME COUNT(*)

---------------------- ----------
                             2062
PER_WS1_gb_UK.pdf               1
PER_WS4_gb_UK.pdf               1
PAY_G52003_ar_SA.pdf            1
ES_company_cert.pdf             1
PER_WS5_gb_UK.pdf               1
FNDATTCH                    14948 - related to Attachments
PER_P11D_gb_UK.pdf              1
PER_SUMM_gb_UK.pdf              1
PER_ADDR_gb_UK.pdf              1
PER_WS6_gb_UK.pdf               1
PAY_G42003_ar_SA.pdf            1
PAY_G32003_ar_SA.pdf            1
FND_HELP                    59198 - related to iHelp
Oracle E Records                2
PER_WS3_gb_UK.pdf               1
PER_WS2_gb_UK.pdf               1

17 rows selected.

Oracle says that it is common to see rows with pdf and rtf files.

To find how much space is actually used by the lobsegments:

select sum(dbms_lob.getlength (FILE_DATA)) from FND_LOBS;

SUM(DBMS_LOB.GETLENGTH(FILE_DATA))
----------------------------------
5,047,907,542

To find the total space allocated in the extents:

select sum(bytes), s.segment_name, s.segment_type
from dba_lobs l, dba_segments s
where s.segment_type = 'LOBSEGMENT'
and l.table_name = 'FND_LOBS'
and s.segment_name = l.segment_name
group by s.segment_name,s.segment_type;

SUM(BYTES) SEGMENT_NAME SEGMENT_TYPE

---------- ------------------------- ------------------
6,040,576,000 SYS_LOB0000040605C00004$$ LOBSEGMENT

Thus, about a gigabyte of this client’s space is allocated but not used.

To find the space used by each program that will not be purged because the expiration_date field is null:

select

program_name,round(sum(dbms_lob.getlength (FILE_DATA))/1024/1024,0) "Size(M)"
from APPS.fnd_LOBS
where expiration_date is NULL
group by program_name order by 2 desc;

PROGRAM_NAME Size(M)

-------------------------------- ----------
FNDATTCH                               3825
                                        546
FND_HELP                                279
ES_company_cert.pdf                       0
PER_WS5_gb_UK.pdf                         0
PER_P11D_gb_UK.pdf                        0
PER_SUMM_gb_UK.pdf                        0
PER_ADDR_gb_UK.pdf                        0
PER_WS6_gb_UK.pdf                         0
PAY_G42003_ar_SA.pdf                      0
PAY_G32003_ar_SA.pdf                      0
Oracle E Records                          0
PER_WS3_gb_UK.pdf                         0
PAY_G52003_ar_SA.pdf                      0
PER_WS4_gb_UK.pdf                         0
PER_WS2_gb_UK.pdf                         0
PER_WS1_gb_UK.pdf                         0

17 rows selected.

So about 4650M will not be purged.

PCTVERSION

MOS Doc. ID: 298698.1, "Avoiding abnormal growth of FND_LOBS table in Applications 11i" explains that "to maintain read consistency, Oracle creates new LOB page versions every time a lob changes. PCTVERSION is the percentage of all used LOB data space that can be occupied by old versions of LOB data pages. As soon as old versions of LOB data pages start to occupy more than the PCTVERSION amount of used LOB space, Oracle tries to reclaim the old versions and reuse them. In other words, PCTVERSION is the percent of used LOB data blocks that is available for versioning old LOB data. The PCTVERSION can be set to the percentage of LOB's that are occasionally updated.

The FND_LOBS table's FILE_DATA LOB column usually gets the data uploaded only once, but it is read multiple times. Hence, it is not necessary to keep older versions of LOB data. It is recommended that this value be changed to "0". By default PCTVERSION is set to 10%. It must be set to 0% explicitly. The value can be changed any time in a running system."

You can use following SQL to find the current PCTVERSION value :

select SEGMENT_NAME, PCTVERSION

from dba_lobs
where TABLE_NAME ='FND_LOBS';

SEGMENT_NAME PCTVERSION

------------------------- ----------
SYS_LOB0000040605C00004$$ 10

Recommendation: Set PCTVERSION to 0 using the command: alter table applsys.fnd_lobs modify lob (file_data) (pctversion 0);

NEXT_EXTENT

To help manage the size of the FND_LOBS table, Oracle’s main recommendation is to set PCTVERSION to 0 and use a reasonably sized next_extent. We checked the LOBS tables for our client, and they had very few extents, so the tables are probably sized adequately.

Oracle also notes that "if using locally managed tablespaces, then the number of extents is not as much of a problem as it was in the past with dictionary managed tablespaces. Since the LOB segments are usually very large, they are treated differently from other columns. While other columns can be guaranteed to give consistent reads, these columns are not. This is because it is difficult to manage LOB data rollback segments due to their size, unlike other columns. So they do not use rollback segments. Usually only one copy exists, so the queries reading that column may not get consistent reads while other queries modify them. In these cases, the other queries will get "ORA-22924 snapshot too old" errors."

To see the approximate size of the extent:

select

min(round(dbms_lob.getlength (FILE_DATA)/1024,0)) "Min Size(K)",
round(avg(round(dbms_lob.getlength (FILE_DATA)/1024,0)),0) "Avg Size(K)",
max(round(dbms_lob.getlength (FILE_DATA)/1024,0)) "Max Size(K)"
from APPS.fnd_LOBS;

Min Size(K) Avg Size(K) Max Size(K)

----------- ----------- -----------
0 59 1173

Oracle says that there is no "magic" number for the extent size and a compromised setting should be used, observing the following:

Extent should not be too small, to avoid constant extent allocation (which would cause HW enqueues in tables with LOBS)
Extents should not be too big, to avoid wasted space
Extents size should be bigger than the majority of the size of one lob segments

ORPHANED RECORDS

The following script from MOS Doc. ID: 963222.1. “Orphaned records in FND_LOBS table when uploading attachments using FNDATTACH form” can be used to check if you have any orphaned records in the FND_LOBS table that have been inserted through the FNDATTACH form.

SELECT COUNT(*)
FROM FND_LOBS FL
WHERE NOT EXISTS
(SELECT '1'
FROM FND_DOCUMENTS_TL FDT, FND_DOCUMENTS FD, FND_ATTACHED_DOCUMENTS FAD
WHERE FD.DOCUMENT_ID = FDT.DOCUMENT_ID
AND FAD.DOCUMENT_ID = FD.DOCUMENT_ID
AND FDT.MEDIA_ID = FL.FILE_ID
AND FD.DATATYPE_ID = 6)
AND PROGRAM_NAME = 'FNDATTCH'
AND EXPIRATION_DATE IS NULL;

COUNT(*)

----------
447

These 447 orphans are caused by a code bug in file (FNDATTCH.pld) which allows the attachment to be loaded into the FND_LOBS table even if the user does not confirm that the upload completed successfully. To implement the solution, execute the following steps:

1. To prevent creation of orphaned records in FND_LOBS going forward, download and review the readme and pre-requisites for:

Release 11i Customers:

Patch 9004099 - ORPHANED RECORDS IN FND_LOBS TABLE CONSUME DATABASE DISK SPACE

Release 12 Customers:

Patch 9454616 - PROGRAM_TAG,UPLOAD_DATE,EXPIRATION_DATE,PROGRAM_NAME ARE NULL

2. Ensure that you have taken a backup of your system before applying the recommended patch.

3. Apply the patch in a test environment.
4. Retest the issue.
5. Migrate the solution as appropriate to other environments.
6. In order to get a data fix to delete orphaned records in the FND_LOBS table that have been inserted using the FNDATTACH form , log a Service Request with Oracle Support.

Recommendation: Apply Patch 9004099 and then log an SR with Oracle Support to get help with cleaning up the orphaned records.

RECLAIMING SPACE

Once you have addressed the records with expired data by running the “Purge Obsolete Generic File Management Data” Concurrent Program, and changed the PCTVERSION to 0, and adjusted the NEXT_EXTENT value to stop the overly aggressive growth of the allocated space, then it will be worthwhile to see if there is a way to reclaim unused space. Read the documentation very carefully and test very thoroughly, though, as these notes describe making substantial changes to the database.

If there is a lot of DML activity in the FND_LOB table, some space may be reclaimed by moving the table to another tablespace and then moving back to the original tablespace. Refer to:

MOS Note: 303709.1 “Reclaiming unused space in APPLSYSD tablespace”

MOS Note: 130814.1 "How to move LOB Data to Another Tablespace"

For additional information, refer to:

MOS Note: 118531.1 How to Compute the Size of a Table containing Outline CLOBs and BLOBs

MOS Note: 386341.1 How to determine the actual size of the LOB segments and how to free the deleted/unused space above/below the HWM

LIMITING ATTACHMENT SIZE

It is also possible to limit the size of attachments. The profile option 'Upload File Size Limit' can be used to limit the size of uploaded attachments.

Recommendation: Research file size limits by reviewing the following MOS Notes:

MOS Note: 605377.1 How To Set A Maximum File Size For Attachments?

MOS Note: 604458.1 How to Limit The Attachment File Size?

Release 11i Extended Support Patching - You'd Better Watch Out! - by Barbara Matthews

2010-11-09T06:42:00.000-08:00

Perhaps you've been seeing the mailnotes, newsletter articles, and even news articles about Oracle's Mandatory Extended Support Patching Requirements. Oracle announced this change in January, 2010, so customers have had quite a while to plan out what they need to do. Perhaps you're thinking "Fah! My company is pretty current, we shouldn't have a problem." Oh, if only that were true. I thought I'd walk you through two case studies of what I've seen, along with a how you can check your status.

To Learn More

If you haven't done so already, please fill out the OAUG's Extended Support questionnaire by November 30th:

http://www.zoomerang.com/Survey/WEB22B7BG2TA85

They'd like to be your advocate, so understanding what Oracle's E-Business Suite customers' status is would help them in discussing issues with Oracle. Also, I'd love to hear more from customers about what they're doing. If you've applied patches, tell me how long it took, how many patches you had to apply, and if you hit any surprises. And if you're still deciding what to do, tell me about that too!

If you'd like to hear more about the Extended Support patches, John Stouffer, who is on the OAUG Board of Directors, will be giving two webinars titled:

What you need to know about Release 11i Extended Support Patching including Tips and Q & A

You can register at::

November 22nd at 1pm EST
or
November 23rd at 1pm EST

Don't miss the opportunity to ask questions and relate your own issues and concerns.

To Check Your Company's Extended Support Status

Let's start with the infamous My Oracle Support Doc. ID: 883202.1. In a nutshell, it says:

To be eligible for Extended Support of 11.5.10, which begins on December 1, 2010, your production E-Business Suite environment must be patched to the patch levels indicated in the table under Section 1, requirements 1 through 6. Also, you must have the patches in Section 2 applied. Section 2 includes a large number of patches for a large number of E-Business Suite modules. You have to apply patches to any module that you have installed, shared, or that is a pseudo module. Also, as we get closer to December 1st, My Oracle Support Doc. ID: 883202.1 may be updated with additional patches. So even if you've gone through this drill, you need to keep checking.

Well that's pretty simple. You might wonder what the fuss is all about, and the answer is three-fold. First, these requirements are mandatory. Oracle has traditionally steered clear of making mandatory requirements, so it's a big deal when they introduce one. And second, this mandatory requirement has dozens of patches that might need to be applied. Those patches will require thorough testing before you can move them into production. Even if you've made it past Section 1 of Doc. ID: 883202.1, you'll likely still have a couple of dozen patches to apply. Last of all, the date: December 1, 2010. It's just around the corner. You'll need to figure out what patches need to be applied, find all the pre-requisites, apply them to a test environment, test them, and then apply them to a production environment. Before year end.

Now, how do you figure out what you might have to patch?

The Manual Way

First, run patchsets.sh. At the top, there are a list of modules that you use. Print the list out. Print 883202.1 out. Get out your highlighter. Highlight all the patches in Section 2 that match up to the modules in your patchsets.sh report - whether installed, shared or pseudo.

The Automated Way

You can use the Patch Wizard to tell you what you need to patch, but you might have to patch Patch Wizard to make that work:

Patch 9803629 includes the following pre-requisite patches: 4125550, 3036401, 3264818, 3263588, 3219567, 3264822, 3263645, 3261254, 3262486, 2614213, 3261243, 4038964, , 3262159, 3412795, 3140000

If you're relatively current on the E-Business Suite, you've probably got many of these patches applied already. Go back to patchsets.sh and take a look there. You'll need to get out your highlighter and highlight those cases where the Running Version column is lower than the Latest Available column. Then compare the highlighted patches in patchsets.sh to this list of pre-requisite patches for Patch 9803629 and see how you're doing.

Next Step

By now you've got a list of patches that you need to apply. If you want to use Patch Wizard - and yes, you should want to use Patch Wizard, because once you've gone through MOS Doc. ID: 883202.1 with a highlighter once, you'll never want to do that again - then you'll need to get organized and plan out those patches. One issue with MOS Doc. ID: 883202.1 is that you've got a list of patches, but guess what - you don't really think those patches won't have pre-requisite patches, do you? So getting Patch Wizard working so it will tell you pre-requisite patches is another great reason for applying all those patches to Patch Wizard.

Following are two examples of client's results. You may be surprised to hear that the client who was patched most current - having all the items in Section 1 up to date - had the most patches to apply.

EXAMPLE 1: We’ll start with a client who is way behind on their patching. They applied Release 11.5.10.2 with CU2 in 2006 and haven’t had much time to patch since then. They’re still using JInitiator, and they’re on RDBMS 10.2.0.3.

From the top part of the patchset.sh Report_11i.txt file:

Limited Report to: APPLFULL and APPLSHAR products

APPLFULL: AK ALR AP AR AX AZ BIS CE CHV EDR FA FII FND FRM FTP GL HZ ICX INV IZU PA PN POA PO POS RCM RG ZPB

APPLSHAR: AD AS AU BIC BIL BIM BIX BOM CRP CUA DT EC ENG FF FLM HRI ISC JTF MFG MRP MSC ONT OPI PAY PER PJM PMI QA SHT WIP XLA

Pseudo Products: ADX AME AML BLC BPA CAC CDR CLE CSK CSZ CTB EDW EWS FTP FWK HCP HCT IGP IGR IPATCH IRC ISX ITA ITM JTA JTH JTO JTP JTT JTU JTY MSX OAM OCM OIE OIR OIT OWF PFT PJR POV RCM TXK UMX

Now let’s look at how far behind patchsets.sh shows this customer. The highlighted modules are ones where there are higher versions available:

Patching Patch Wizard

Now let’s start with what it will take to get Patch Wizard patched for this client:

4125550 - 11.5.10 CU2 for ATG Product family - OK
3036401 - Mini-Pack 11i.HZ.L – OK, on N
3264818 - Patch 11i.UMX.H – OK, on H
3263588 - Patch 11i.XDO.H – not seeing XDO, see below Patch 3412795
3219567 - Patch 11i.TXK.B Technology Stack Minipack B (also in 11.5.10)- OK, on B
3264822 - Patch 11i.CAC.B – OK, on C
3263645 - Patch 11i.AK.G – OK, on G
3261254 - Patch 11i.ALR.G – OK, on G
3262486 - 11i.JTA.F – OK, on F
2614213 - AME PATCH :DELIVERY OF GA AND RULE PRIORITY FUNCTIONALITY – will need to check if this is applied
3261243 - Patch 11i.EC.G – OK, on G
4038964 - Minipack 11i.AD.I.1 – OK, on AD.I.4
3262159 - Patch 11i.FND.H – OK, on H
3412795 - ADSPLICE PATCH FOR XDO – not seeing XDO, so need to apply this
3140000 - Oracle Applications Release 11.5.10 – OK!

So the good news is, since this client is on Release 11.5.10.2 CU2, they’ve got almost everything that they need to make Patch Wizard work. They’ll have to check into the AME Patch 2614213 and apply Patch 3412795 and Patch 3263588 to get XDO going.

Reviewing Section 1 in 883202.1

These are the seven requirements in Section 1, and our client has issues with the yellow highlighted ones:

Whew! That’s a lot of patching! I’m not going to list the patches from Section 2, just suffice it to say that there are 37 additional patches, and those patches likely have pre-requisites. So, if you were this customer, what do you think? Could you upgrade to RUP 7, upgrade your database to at least 10.2.0.4,upgrade to Forms6i Patchset 19, upgrade your users’ from JInitiator to JRE and apply at least 37 additional patches by December 1st? That’s the challenge!

Example 2: A client who is much more current on their patches

For this example, the client is running RDBMS Version 10.2.04, and they’ve applied ATG RUP 7, which brought them up to date on a lot of installed, shared and pseudo modules. They’ll need to check if Patch 2614213 - AME PATCH :DELIVERY OF GA AND RULE PRIORITY FUNCTIONALITY is applied, but otherwise they’re set to apply the Patch Wizard Patch 9803629. Once they apply Patch 9803629, they can double check the list of patches identified, including pre-requisites, and begin planning their test cycle.

Limited Report to: APPLFULL and APPLSHAR products

APPLFULL: ABM AK ALR AMF AMS AP AR ASF AS ASL ASO ASP AST AX AZ BEN BIC BIL BIM BIS BIV BNE BOM CCT CE CHV CRP CSC CS CSI CSS CUG CZ EAA EC ECX EDR ENG FA FEM FII FND FRM FV GHR GL GMA GMD GME GMF GMI GML GMP GR HRI HXC HXT HZ IBA IBC IBY ICX IEO IES IEU IEX IGI INV ISC IZU JTF JTM ME MRP OKC OKE OKI OKL OKS OKX ONT OPI OTA PA PAY PER PJM PMI PN POA PO PQH PQP PSA PSB PV QA QOT QP QRM RG SSP WIP WSH XDO XNI XTR

APPLSHAR: AD AMV ASG AU BIX CN CSD CSF CUA DT FF FLM IBE IEM MFG MSC PSP SHT XLA

Pseudo Products: ADX AME AML BLC BPA CAC CDR CLE CSK CSZ CTB EDW EWS FTP FWK HCP HCT IGP IGR IPATCH IRC ISX ITA ITM JTA

This client passed the Section 1 requirements in MOS Doc. ID: 883202.1 with flying colors:

Section 2, on the other hand, presented quite a few patches:

There are at least 44 installed, shared, and pseudo modules in this client’s production environment that need patches tested and applied, with at least 67 patches to be applied. The reason this client has more patches to apply from Section 2 is that they have more modules licensed.

This client has decided to apply the extra patches, and is in the middle of a patch current exercise right now. So far, they report that they haven’t hit a lot of problems, but other clients note that applying patches to certain modules has required extensive additional research and time, so your mileage may vary. Also, this client says that there is no way they can get all of these patches tested and installed by December 1st, 2010.

OOW Blogging: The Older I Get, The More Behinder I Get

2010-10-13T07:29:00.000-07:00

The last time I went to Oracle OpenWorld was, I think, the first year that Oracle called this huge conference Oracle OpenWorld. So, I'm thinking, maybe more than 10 years ago. I hated it. There were a kajillion people thronging around, and every session that I tried to go to was filled past capacity. So, I spent most of my time sitting on the floor outside of rooms, contemplating my database, thankful to at least be away from it, but a little put out.

So I have to say, I never wanted to go back.

But this year, I got a bee in my bonnet and decided that I wanted to see hands on demonstrations of some of Oracle's products. So I cast about for a way to attend, and got permission to attend as an Official Oracle Blogger. So first and foremost, let me just say that I appreciated the opportunity very much. Thank you to Oracle and to Larry Ellison. I wouldn't have been able to come without the free pass.

So now I'm writhing with guilt, because I didn't blog. I didn't blog about what I intended to attend before the conference started. And I didn't blog about what I was seeing while I was there. And I didn't blog about what I saw afterwards. Well, now I'm blogging.

I have many excuses, of course. Before the conference, I was working very hard on preparing for the conference (getting all my work done). And during the conference, I was staying in Oakland, so I had to ride the BART every day, which is a big deal for someone who lives in the countryside and never rides these modern forms of transportation. And then there were the cats. Yes, I stayed with my brother and sister-in-law, and so all my non-blogging during the conference was their fault. They have these three cats, who loved me very much. And I forgot to prepare for the cats, so I arrived without a bag full of antihistimine. So one day, I fell asleep on the couch, when I should have been blogging, and one of the cats decided to hunker down with me, and that was the end of any blogging potential during the conference.

On the way home from the conference, the influence of the cats was so great that the man sitting next to me on the plane offered to pray for me, as I was clearly in distress, with my head in my hands, and my hands full of Kleenex. I took him up on it, and afterward he asked if it had helped. And I had to say, "Err... no, not really, but I thank you very much anyway, because it certainly didn't hurt."

For two + weeks after the conference, I did my level best to hack up an extra cat for my brother and sister-in-law. Finally, I am still coughing, but "not so much". My mother has skipped church for two solid weeks because she doesn't want people to worry that we will give them tuberculosis (she has a wee chronic cough herself). When I cough, I sound like a seal, or maybe a donkey braying. You cannot help but feel sorry for me.

Before I move on to my official recounting of what I learned, I wanted to say that the conference itself, in terms of organization, is sooooo much better than it was the last time I attended 10+ years ago. Attendees are strongly encouraged to sign up for sessions ahead of time. So I carefully went through the list of presentations and signed up for all the hands-on sessions I could find. And, although there were tens of thousands of people at the conference, I never had that standing room only feeling, so somebody must have handled the room selection very carefully. I was not overly fond of all the walking I had to do, but it's hard to complain, since walking from Moscone South to Moscone West to the Marriott meant there was a vast quantity of sessions. There wasn't any time when I couldn't find something interesting to attend. And, as for the walking, I needed it, and once I had proper walking shoes (courtesy of my sister-in-law), I managed just fine.

So, if you're wondering if it is worth it to attend OOW, in my opinion, it definitely is. I still like the smaller Collaborate conference better, but that's primarily because my focus is on the E-Business Suite of Applications, while OOW has a much broader focus. But if Larry invited me back next year, I'd go without hesitation.

So, Larry, thank you again for letting me attend your conference. It was a wonderful experience, I learned lots of things, and now I'm going to blog about what I learned.

Collaborate 10 OAUG Upgrade SIG Meeting by Barbara Matthews

2010-05-10T10:48:00.000-07:00

The OAUG Upgrade SIG hosted a meeting at Collaborate 10 to discuss upgrade issues from E-Business Suite Release 11i to Release 12.

The Panel

- John Stouffer, Independent Consultant, Past Chair of the Upgrade SIG, Moderator
- Lester Gutierrez, Oracle EBS Performance Group, works on upgrades and reducing downtime
- Udayn Parvate, EBS Release Management Team, works on building and packaging the E-Business Suite software. Udayn performs installation and upgrade activities, defines and enforces standards for packaging and delivery with development teams. He works to get upgrade issues resolved so E-Business Suite customers' experience is as smooth as possible.
- Sandra Vucinic, Vlad Group, Inc., Chair of the Upgrade SIG
- Floyd Teter, Jet Propulsion Labs, functional guy on the panel
- Steven Chan, Director, Applications Technology Integration, Oracle Corporation
- Michael Rulf, Executive Director - Product Development, AT&T Hosting and Application Services (H&AS)

The Questions and Answers

We are going to upgrade from Release 11.0.3 to Release 12. We are currently running on Sun Solaris and will switch to a Linux Intel environment. Our architecture team is wondering how to do performance load simulation so we know how much hardware we need to buy to run the upgraded E-Business Suite environment. Our database is 700 gigabytes, and we have 2500 users.

- First, you need to get a sense of how many concurrent users you have, and what those users are doing. While you may have a total of 2500 users, those users probably aren't all logged on at the same time, and they likely aren't submitting data and hitting commit at the same time. The limits will not be software, but will be hardware limitations based on the degree of throughput, and read and write operations per second. The architecture team needs to understand the product mix (modules), the read/write workload, and the concurrent manager workload, and then bring in the hardware vendor to help build out a test environment. You can also use load simulation software like Mercury Interactive. Oracle has a Test Starter Kit (see http://blogs.oracle.com/stevenChan/2010/04/ebs_1211_tsk.html) for E-Business Suite Release 12.1.1 that can be helpful, but you need to choose carefully when you put together your test environment.

- As you purchase your hardware, make sure you are positioned to be scalable, so you have room to grow if you need to. You might find, for example, that when you do the first close on the new system, you need to add more hardware to deal with the extra load. If you can move to parallel concurrent processing, you can add another node easily. Given the size of your environment, you should research and understand the options available for using load balancing and shared appl tiers (see Parallel Concurrent Processing Failover and Load Balancing of E-Business Suite Release 11i and Release 12 by Mike Swing at TruTek).

- As far as hardware is concerned, we are starting to see where the Intel world is supporting more processor cores, and the hardware is hot pluggable. Be careful not to skimp on the chassis, but populate it with the number of processors you want. We're about to see a big jump in the industry from 4 core to 8 core. You should try to pick a hardware solution that allows you to make that move if you need to. Note that there's a tradeoff to buying more processing power than you need - if you overload your processor, your software fees will go up, so you have to strike a balance.

We upgraded to release 12.0.4 and saw a 25-30% increase in database size. Is there a similar increase with the 12.0.4 to 12.1.2 upgrade?

TCA, SLA, and E-Business Tax added a lot to the database size, but you've already bit the bullet on those modules, so no, you shouldn't see a significant increase.

We are currently running Release 11.5.10.2. Is there a recommended upgrade path?

You should upgrade to RDBMS Version 11gR2 and then Release 12.1.1, and then apply the 12.1.2 RUP. We've "borrowed" a wonderful diagram from Steven Chan's blog of what your E-Business Suite Upgrade Roadmap is:

Is there a plan to offer a gui screen for the Mobile Supply Chain module? Currently it uses a telnet session.

It is a high priority with the User Experience team, but we aren't sure when it will be released. The Release 11.5.10 gui version, according to someone in the audience, was too slow, so it wasn't adopted by many users.

Do people typically upgrade or reimplement?

Upgrading is less expensive, and much better supported by Oracle. Because of the nature of reimplementations, the path isn't as clear because we all have different reasons for doing it. It is much more difficult to determine what post upgrade patches to apply, and how to do validation testing. Reimplementation customers will have to blaze their own trail.

Is bifurcation still recommended?

For Release 12, the new term is Upgrade by Request. Upgrade by Request allows you to upgrade part of your data during the upgrade, and then upgrade the rest of your data after your upgrade completes. Modules that can use Upgrade by Request include Financials and Procurement, projects, Supply Chain Management, and CRM. You can read more about Upgrade by Request in Appendix G of the Release 12 Upgrade Manual.

Are there any recommendations for preparing for a Release 11.5.10.2 upgrade to Release 12.1.2?

- See Sandra Vucinic's presentation, Get Ready for EBS Release 12.1! Tasks to Complete Now to Ease R12.1 Upgrade Process. Here's a list of tasks from her presentation:

- Upgrade database to 11gR1 or 11gR2
- Convert tablespaces to OATM model
- Evaluate impact of R12.1 on customizations and extensions
- Introduce BI (XML) Publisher and JDeveloper
- Introduce Web ADI and Report Manager
- Archive and purge EBS 11i data
- Convert from JInitiator to Native Sun JRE - _21 is recommended
- Integrate Discoverer Server Release 10g with EBS 11i - use Discoverer 11g if possible as Discoverer 10g is desupported at the end of 2010
- Configure Oracle iAS Release 10g for external apps (SSO, OID, Portal) and integrate with EBS 11i
- Position for high availability and scalability
- Evaluate and complete platform change based on ROI

- RDBMS 11gR2 is a major architectural change in the nature of the cluster software. DBAs need time to understand the fairly steep learning curve. The many new features may help justify doing the database upgrade separately from the E-Business Suite upgrade so DBAs can come up to speed on the technology.

- In terms of testing, have you catalogued all of your customizations? How are you going to train your user community? What tool will you use to develop training guides?

- Read the 11gR2 Upgrade Companion, and be sure to stay current throughout your upgrade on new patches. There's already a RDBMS Version 11.2.0.1.1 patch which fixes a lot of issues.

A user related his story of upgrading from Release 11.5.10.2 to Release 12.1.2. During the upgrade the patch stuck on a program, they waited a day, and got a pre-upgrade patch that needed to be applied before starting the upgrade. He noted how frustrating this was, and asked if there is any My Oracle Support note that tells all performance and serious functional issues in the upgrade.

The official My Oracle Support documents are the Release 12 Upgrade Guide, the Release Notes, and the NLS Release Notes. When Oracle encounters issues, they update the Release Notes. They may also release additional patches, including Critical Update Patches (CUP), which include a consolidation of fixes. Users must actively monitor My Oracle Support for additional patches and alerts while upgrading. Another option is to look at the Release 12 Forum, a very active, live, real time forum. Often solutions can be found there before logging a problem in My Oracle Support. Check out the forum at: http://forums.oracle.com/forums/forum.jspa?forumID=395&start=0

How close are we to desupport on Release 11.5.10.2?

Premier Support ends at the end of November, 2010. The extra cost for Extended Support is waived until November, 2011. Extended Support ends in November 2013. Premier support is what you have today, and includes certifications with Oracle products and other products. If a new MS Windows client is released, with Premium Support, Oracle will certify to that. Extended Support, however, will not include third party certifications, so if a new service pack for Windows 7 is released after November, 2010, Oracle is not bound to certify to that service pack. Odds are, they'll give it a try, but they are not likely to produce new patches. The emphasis at Oracle once Extended Support kicks in for Release 11.5.10.2 will be on Release 12.

This point is very important - E-Business Suite customers use a variety of other products besides the E-Business Suite on their computers. Once Extended Support starts, customers may find themselves needing to upgrade because of some other product that they use, but unable to do so because of certification issues with Windows or JRE or other products.

When you move to R12, JInitiator is not certified and won't work on Windows 7 or Vista. You will need to migrate to the native SUN JRE client. Don't have to worry about java conflicts between applications.

I am new to an organization that uses grants, contracts, and projects. In the past, these were considered outliers that kept us from going to the latest release. Will these modules impact our ability to go to Release 12?

Panel members pointed to several companies using those modules who are upgrading, and said they have not hit any big issues.

An audience member said that she had put together a plan for her company based on what she has heard at the conference and wanted to make sure they were heading in the right direction. They are running RDBMS Version 10.2.0.3 with Release 11.5.10.2, they are already on JRE, they have Discoverer and ADI/GL already implemented with the web, and are moving to RDBMS 11g using OATM with a 2 node shared $APPL_TOP and the latest Release 12 (currently 12.1.2). Is that where her company should be heading?

The panel agreed, her approach is correct.

What features of RDBMS Version 11gR2 would be useful to the E-Business Suite Applications?

You can save a baseline of your production RDBMS 10g performance, and 11gR2 will tell you that your current execution plan is fine, or that you need to run a different plan. You won't lose any performance by going to 11g. This feature is built into 11g. If you run into a performance problem, you can fall back to the 10g plan to get immediate relief while working with Oracle Support to get a more permanent fix.

Conclusion

All in all, it was a fun meeting with lots of questions and interesting answers. As one might expect, the OAUG Upgrade SIG strongly urges you, if you haven't done so already, to get ready to upgrade to Release 12. There are exciting features in RDBMS 11gR2 that will benefit the E-Business Suite Applications even if you don't upgrade to Release 12. The path to upgrading is well-laid out and tested, and the software is now very stable. If you have questions about the upgrade, feel free to contact the OAUG Upgrade SIG. Just drop a note to barb at oncalldba dot com

Collaborate 10 OAUG Database SIG Meeting by Barbara Matthews

2010-05-10T10:29:00.000-07:00

The OAUG Database SIG held a lively meeting at Collaborate 10 in Las Vegas. Steven Chan, Senior Director in Oracle's Applications Technology Group, spoke at length about the nuances of Oracle's Lifetime Support policy. You can (and should!) read the details about Lifetime Support at http://www.oracle.com/support/lifetime-support-policy.html.

Understanding just what is included with Extended Support and Sustaining Support, and at what price, is important for customers to understand. You can also sign up for My Oracle Support notifications to receive automatic emails when the LSDs (Lifetime Support Documents) change.

Steven explained that E-Business Suite Release 11.5.10 Premier Support lasts six years from November 2004. That means that at the end of November, 2010 - that's right, this year - Premium Support ends. To stay supported on Release 11.5.10 after that, you'll have to pay a premium on support costs to run in "Extended Support". Steven strongly recommended not running in production in Extended Support unless you really need to. He described Extended Support as a dangerous place to be.

And here's where the nuances come in. With Extended Support, you can still log a P1 problem - but there's no guarantee that the resolution will come quickly - it might take months. Even within Premium Support, Oracle supports only the current and previous database releases for 12 months after the current database has been released. That's a subtle point that could cause big issues for customers - if you are running RDBMS 10.2.0.4, you're supported for only 1 year now that 11gR2 is available. If you are running RDBMS 10.2.0.3, thinking you are supported based on what you read on the support page, you aren't - that support stopped in February, 2009. These policies override the E-Business Suite support agreements.

One technical issue brought up at the meeting concerned Sun JRE. Sun JRE is required on Windows desktops for users to access the Applications. Up until JRE 1.6.0_17, the software worked fine. However, Sun introduced changes that broke session management and ordering with JRE 1.6.0_18. The recommendation at the meeting was that if you were still on JRE 1.6.0_17, you should not apply versions 18, 19 or 20, because they all have significant issues for E-Business Suite users. With automatic updates on PCs, you can imagine the issues that this might cause. Also, certain customers, particularly government customers, are required to stay current, so those customers had no choice but to upgrade to the latest version of JRE, even though it had these two major bugs.

Now add one more point - JRE 1.6.0_20 was released to correct a significant security issue that occurred between JRE 1.6.0_17 and JRE 1.6.0_20, which means that if you're on JRE 1.6.0_17, you're ok, but if you are on JRE 1.6.0_18 or JRE 1.6.0_19, you need JRE 1.6.0_21. For military organizations that are required to stay within a certain number of releases of the most current, they may have no choice but to upgrade. Steven commented that while this issue was going on, there really was no reassuring place for an E-Business Suite administrator to be right now.

Mark Farnham suggested that if you're a non-DOD compliant organization with pretty good campus-wide security, and you're a low enough target value, then you can wait for JRE 1.6.0_21, which is expected shortly. The PCI (credit card) industry has a similar issue to military organizations. They have to apply all vendor security patches within 30 days of their release. The good news, according to Mark, is that the really mercenary hackers aren't interested in most organizations.

The recommendation, therefore, is that as soon as a release becomes available that doesn't have the functionality issues, all E-Business Suite users should seriously consider moving to that release. You can read more about this issue on Steven Chan's blog at:

http://blogs.oracle.com/stevenChan/2010/04/warning_e-business_suite_issues_with_sun_jre_160_20.html

Another issue with versions comes with third party vendors. If third party vendors don't stay current on JRE versions, then users may find themselves working with releases that have bugs or security issues. Agile and Siebel were two examples of vendors with this issue. Steven said that the only thing customers can do in this case is apply pressure on the third party vendors. The advantage of staying current with JRE, for example, is that each new release brings new memory management capabilities, better security, and bug fixes, so, with the exception of the temporary issues with JRE 1.6.0_18 - 1.6.0_20, staying current is the way to go.

Yet another example of the need to understand the nuances of Oracle support agreements lies with the 9i Application Server used with Release 11i and the 10g Application Server used with Release 12 applications. The double edge to that sword is that Application Server 10g has its own application lifecycle, and the Application Server 10g obsolescence life cycle overrides the E-Business Suite life cycle.

One of the questions from the group was what Oracle is doing to make this situation better. Steven said that Oracle is testing better and more, taking more time to test and running more tests. Now that Sun is integrated with Oracle, there's a better chance of working better together. The Oracle team now gets early shipments of Application 10g Patchset 3, and similarly will get early builds of JRE releases at some point.

One very strong point that Steven highlighted is that Oracle can only go so far in their testing. He strongly recommended that if you are testing a combination of software that isn't documented by Oracle, then it is your job to test heavily, as there are too many possible combinations for Oracle to test.

Randy Giefer, from Solution Beacon, asked if Sun Java might merge into Oracle's CPU cycle. Steven said that while this was a nice idea, he wasn't sure it would ever happen, as we can't really afford to wait for up to 3 months for new changes. He also said that the JRE crowd marches to a different beat because of military requirements.

Also covered in this meeting was the need to stay current with Oracle's quarterly CPUs (Critical Patch Updates). Oracle has improved their CPU process by making the CPUs cumulative as of January 2010 for the Applications users. Where in the past, if you wanted to patch current you would have had to apply each CPU, now when you apply the latest CPU - currently the April 2010 CPU - you're set and don't need to apply any others until the next cumulative quarterly CPU comes out in July 2010.

One additional recommendation discussed was the need to periodically review your company's Master License Agreement. Customers need to be aware that the Oracle Account Manager can't authorize anything that flies contrary to the Master License Agreement. An example - implementing custom objects can eliminate a customer's ability to run using a runtime license. One subtlety, however, is that if Oracle directions say that you have to create an object (there are instructions, for example, that tell you to create custom indices for some of your GL objects), then you are ok and are not violating your E-Business Suite license.

Customers have run into similar issues with partitioning. If you are a licensed EBS customer and Oracle introduces a new product requirement, they can't force you to buy a new license. This caveat is driven by mandatory upgrades of the E-Business Suite that would otherwise have forced you to upgrade the technology. On a new license, for example, customers have to buy the partitioning option when licensing the database, because the E-Business Suite uses partitioning. Customers that upgrade, however, don't have to buy the partitioning software as long as they don't partition any data themselves.

Steven mentioned one area with exciting upcoming changes - for those customers who have considered Active Data Guard to create a mirror of an E-Business Suite database for hardware or software failure, it would be really valuable to be able to use that database for reporting. Currently, the database simply sits, unused, unless a failure occurs.

For Active Data Guard to work, you would have to break the mirror to query against the database, but Steven said that Oracle is working on a series of patches that would allow you to run reads against the real-time copy of the database. This functionality will be included with Version 11gR2, and there may be a port for 11gR1. The one catch is that not all reports will work in this environment. Steven asked that if anyone had E-Business Suite reports (provided by Oracle, not custom reports written by customers!) that they would especially like to be able to run on Active Data Guard, that they should send Steven a note with a list of those reports so that he could ensure that they were tested.

That said, we leave you with two pieces of valuable information - first, Steven's blog, http://blogs.oracle.com/stevenChan/about.html, and second, his email, steven dot chan at oracle dot com.

An Inquiring Mind Wants to Know (More About Reimplementing vs Upgrading) by Barbara Matthews

2010-05-10T10:04:00.000-07:00

At the end of the Reimplement vs. Upgrade Panel at Collaborate 10, someone asked the panel ”Do third party products that claim to fix the issues that might make you decide to reimplement actually work?” The issues include changing the chart of accounts, calendar, and organization structures, or consolidating multiple instances to a single global instance with a shared service center.

The question went unanswered, because no one on the panel had firsthand experience. Since Floyd Teter said “Reimplementing is like getting a root canal without anesthetic in the lobby of the IRS while you're waiting for the audit,” and others on the panel agreed, I decided to do a little digging around. If you can avoid reimplementing the full E-Business Suite and migrating your 11i data, based on everything that was said by the panel, then it would be worthwhile to do so.

So I searched for a software vendor and came across eprentise. I started with Skip Straus , an ex-Oracle Consulting Practice Manager and now an eprentise salesman, and asked him lots of questions. Then I read a paper that he wrote called Why Reimplement? I really wanted to attend the eprentise presentations at Collaborate on global instance consolidation and upgrade vs. reimplement, but that didn't work out. I spoke to Skip again over the phone, asked more questions, and finished up by looking at the eprentise website.

I also asked around to see if my consulting colleagues had any opinions.

The biggest concern raised by consultants was whether Oracle would support a customer that used eprentise transformation software to change Oracle EBS’s "implementation-time configurations." According to Skip, Oracle does not support third party tools, nor does it support the conversion scripts that you would write yourself to extract and transform the data. The assorted eprentise solutions change the data content or format (such as numeric to alpha), but not the database structure. The result is consistent and correct. He said that eprentise has lots of customers who have made the move, and they haven't had any customer report that Oracle wouldn't support them. The eprentise software changes and converts the data; the company supports the conversion and will address any issues. That does not violate any Oracle support agreement.

Also, if you're going to reimplement because your data is bad, your choice is to clean up your data anyway, or abandon it since it’s untrustworthy. If you are reimplementing because you want to change your chart of accounts, well, you'll still be changing your chart of accounts as part of the reimplementation. So it would seem like fixing the data over on your Release 11i side, making the changes to base setups in the E-Business Suite that are not easily changed, would be a good way to go. I also like the idea of changing your data first, and upgrading second, rather than trying to do both at the same time as part of a reimplementation.

Migrating all your data manually seems like a painful way to go rather than using Oracle's well-tested upgrade path. I understand the panelists who said it would take a lot more time and money to reimplement and migrate your data.

I'll leave you with these questions:

1) If you've used a third party vendor's software to avoid reimplementing, whose product did you use?
2) Did it work?
3) Were the tools easy to use?
4) Would you recommend this path to others?
5) Do you have any caveats?
6) Did you run into any issues with Oracle Support?

Drop me a line at editor@trutek.com, as I'd love to hear more about this.

Collaborate 10 Release 12 Reimplement vs. Upgrade Panel by Barbara Matthews

2010-05-10T08:18:00.000-07:00

The Panel

Panel members included Sandra Vucinic, Vlad Group, Inc., John Stouffer, Independent Consultant, Floyd Teter, JPL, Stephen Horgan, Oracle Corporation, Kyle Harris, Oracle Corporation, Mike Swing, TruTek, and Alyssa Johnson, Solution Beacon.

The Questions and Answers

What are the main decision points for upgrading versus reimplementing?

- If you are thinking of changing your accounting structure, you may have to reimplement. Reimplementing can offer a good opportunity to make changes to the accounting setup manager, subledger accounting, and the chart of accounts, and implement additional ledgers. Other reasons that might drive a reimplementation include if there is significant bad data in the existing environment, if the customer has lost their customization history, or company wide consolidations, mergers and acquisitions.

- The biggest issue with reimplementing is money. When you upgrade, you can expect to spend 20-25% of your original implementation costs. Reimplementation costs go way up; close to the original implementation costs. Floyd Teter suggested that if you can squeeze by with an upgrade, do the upgrade. If something dramatic has changed with the way you model your business, then reimplement. Floyd said that reimplementing versus upgrading was strictly a cost/benefit tradeoff.

Do reimplementations take longer than upgrades?

- Floyd said that reimplementation is always going to take longer. He suggested tacking on 40% to the schedule in terms of effort, but it could be even more, depending on your implementation.

- If you haven't archived or purged your data in a very long time, and have a lot of historical data, how does that drive the implement versus upgrade decision? Can you update selected data?

- You should archive as much data as you can before your upgrade process starts.

- You can also bring over only the data you want, and use your Release 11i instance to reference historical data.

- You can use Oracle's Upgrade by Request process to pick and choose how much data to bring over. Oracle provides a lot of flexibility, but you have to time it and make sure it makes sense. With SLA there is a pre-upgrade patch that lets you control how much of the data you upgrade. If you have 10 million debits and credits, they will all get updated, but you can use Upgrade by Request to update some now and the rest later. You can upgrade the last fiscal year of your data, and it will ensure you have at least 6 months worth of data when you go live.

Are there any lessons learned from upgrading?

- The technological architecture changes, along with E-Business Tax, SLA and TCA caused a big increase in size with Release 12. TCA alone went from 30 tables to 300. You should expect to need an additional 20-25% of disk space for Release 12.

- With the original upgrade to Release 12, AP changed considerably and there were a number of functionality issues that came up. AP is in much better shape since Release 12.0.4, although 12.0.6, the most current release, is also the terminal release for 12.0.x. Performance has improved and there are less errors. Oracle has a proactive information center on My Oracle Support that has information about AP. In many cases, you can find solutions without even logging an SR. AP in Release 12.1.2 is as stable as the other subledger modules.

How does customization impact the decision to reimplement or upgrade?

The consensus was that customization impacts need to be dealt with for both reimplementing and upgrading, and that customizations shouldn't have more impact on one method over another. For both reimplementing and upgrading, customers should consider the following:

- No matter what you decide, those customizations are in your business now because they serve a purpose and are addressing business needs. From that point of view, you need to decide whether the business process gap is still there or not. You need to figure this out whether you plan to upgrade or reimplement. The good news is that sometimes customers have been able to eliminate customizations thanks to new features in Release 12.

- You should catalog your customizations. You should know why you put those customizations in and map it to Release 12 functionality to see which customizations you can retire, and then retire what you can. By doing this groundwork, your upgrade or reimplementation will be that much easier.

- Oracle Consulting offers in its Advanced Customer Services a CEMLI Catalog Service that provides an inventory of your CEMLIs (Configuration/Customization, Extension, Modification, Localization, and Integration). These are scripts, and don't appear to be generally available except through Oracle Consulting. But they sounded like they would be quite handy. You can read more about them at http://www.oracle.com/support/advanced-customer-services/cemli-services.html - They are available as part of a "free" Oracle Insight engagement.

The Gorilla in the Room: We're thinking about reimplementing next year. Should we do that or wait for the Fusion Applications?

These were the pearls of wisdom from our experts:

Sandra - Consider Release 12 as a milestone in your journey. It is stable. There are live customers on it. It is a known animal. Doing nothing leaves you with less options. R12 is your milestone to get to Fusion. A lot of great business functionality is available now. With desupport for Release 11i coming in December, 2010, you need to get to Release 12, rather than wait for Fusion.

Mike Swing - If you wait to upgrade to R12, you are putting your production system at risk of being forced to upgrade in a hurry to resolve an issue that can't be resolved in R11i.

Alyssa - Don't wait, you won't regret it.

Floyd - Reimplementing is like getting a root canal without anesthetic in the lobby of the IRS while you're waiting for the audit. Upgrading is always the better choice if you can.

Kyle Harris, from Oracle, pointed out that you may not have a path from Release 11i directly to Fusion. Floyd explained that there's no guarantee that Oracle will offer a direct path from Release 11.5.10 to Fusion 2 or 3. The current direct path offering is for Release 11.5.10.2 to Fusion 1. So if you think you're going to sit on Release 11.5.10 and wait for a fully functional Fusion Application, don't. 80% of E-Business Suite Release 11.5.10 customers are probably not good candidates for Fusion Release 1. Fusion Release 1 will not include manufacturing, for example.

Stephen - Oracle Support will support the upgrade and not the reimplementation. Customers need to really make sure they can't upgrade before considering reimplementation. This point was made at other conference presentations as well - Oracle has limited resources for testing and must focus on what the vast majority of customers will be doing, which is upgrading. Reimplementation issues will vary by customer, depending on their situation, so Oracle Support will certainly try to help with issues, but the more thoroughly tested path will always be the upgrade path.

In the past, when we went from Release 10.7 to Release 11i, Oracle had good numbers on the expected downtime window given certain factors. What is a typical idea of what you can expect for the downtime window?

Mike Swing suggested that the upgrade from RDBMS Version 10.2.0.3 to 11.2.0.1 could take 8-12 hours, with the applications upgrade taking 12-15 hours, plus 1 day of testing; so about 3 days. To get that 3 day downtime window, the other panelists suggested you would need to consider the following:

- Test your hardware environment. In most cases you will bring in new servers. Do as much as you can ahead of time. Consider adding CPUs that you might not use moving forward. Can you lease additional CPUs for the upgrade period? Can you borrow CPUs from other hardware?

- Brainstorm ways to improve performance with your third party vendors and consultants.
Use Upgrade by Request to process the most necessary data during the upgrade, and then complete processing the rest of the data after go-live.

- Consider taking the database upgrade off the table. All new releases of the database have been certified. Mitigate the risk. While you do have to test twice if you do the database upgrade separately from the applications upgrade, the testing is different for the database upgrade, where performance considerations are the main issue, and the Applications upgrade testing for R12 is largely functionality testing.

- You won't know until you test it, then, after you test it, try to figure out how much time it will take and what you can do to reduce the time. There are lots of documents about how to reduce the downtime on My Oracle Support.

Summary

All in all, the consensus from the panel was that if you can avoid doing a reimplementation, you should, due to issues with cost, support, and time. For those customers who are waiting for Fusion, the panel concluded that if the functionality provided in Fusion Release 1 doesn't match your business needs, then you should move to Release 12 first, and wait for Fusion to catch up with your requirements in a future release.