Posts belonging to Category SSIS



Connect Item on Import/Export Wizard

I like using raw files in SQL Server Integration Services (SSIS).  They are a quick way to build data repository that will be used else where in a package or for use with a separate package.  In a recent project, raw files were used to move client data around in a new import process that consolidated business logic and improved performance. 

As much as I like raw files, there is a place in SQL Server where the love I have for raw files is lacking.  This is with the Import/Export Wizard in Management Studio.  If you look through the wizard, you’ll find Excel love, Access love, and even some Oracle love.  But there isn’t an option to import or export raw file data.

 

Stop the Madness

Susan Powter said it best, “Stop the Madness”.  Ok, maybe this isn’t madness.  But it sure is irritating when I have to write an SSIS package from scratch to import a raw file to my sandbox database to see what data is in it that is causing issues.  If it were any other data source or destination, there wouldn’t be a need to do that extra work.

Is this extra work very hard?  No, of course not.  But this is a barrier to using this format.  I’ve been pushed away from them before when explaining the need to build a package to view raw file data.  When other formats can be imported with just a few clicks of a button.  The five to ten minutes to stop and build the package and import the data can add up in some situations.  And compating it to the minute or so to use the wizard, this can be a significant time savings when developing and testing SSIS packages.

There is certainly no sense in complaining without trying to fix.  So here it goes…

If you think this idea is kicking, special, or worthy of your support – please go out and vote up the Connect item that has been submitted on it.

Tap, Tap, Tap – SSWUG Conference Starts Today


ssccdl_couch-laptop1_250x20
The SSWUG Ultimate Virtual Conference starts this morning.  Are you planning to take advantage of this opportunity?  The Rock Starspeakers are waiting in the channels, anticipating you every move.  But, have you registered?

It’s not too late.  If you can’t get to the sessions today, there is always tonight to go through and view them.  Just check out what the conference includes.

  • More than 75 technical sessions
  • New! Live sessions track – all live sessions, with speaker Q&A/interviews at the end of the day
  • On-Demand access to sessions for 45 days – miss a session? No problem.
  • Chat, Twitter Integration, SKYPE integration for Q&A
  • Experience-based learning – find out what you need to know from people that are using the technology every day
  • These are NOT sales presentations.
  • 6-Month SSWUG.ORG Membership (or membership extension), included!
  • ALL-ACCESS Pass: SQL Server and Business Intelligence with related SharePoint and .NET technologies – all included, one low price
  • Great vendor hall – learn about the best tools, technologies, publications and partners out there for your shop

Hope in and join the conference!  I’ll be there after my morning meeting.

Are You Registered?


ssccdl_couch-laptop1_250x20
It starts next week,the SSWUG Ultimate Virtual Conference on October 21-24.  This is an excellent opportunity to get the benefits of heading to a conference full of Rock Star speakers while being able to view the content from the comforts of home or your desk.

Have you registered?

If you haven’t registered yet, I really recommend that you do.  Heck, you can even use this discount (SPJSUVC09) to save a little on the registration.  But I’ve told you what to do, have I told you why.  Well, let’s take a look at some bullet points that describe the conference: ‘Cuz we all like bullet points.

  • More than 75 technical sessions
  • New! Live sessions track – all live sessions, with speaker Q&A/interviews at the end of the day
  • On-Demand access to sessions for 45 days – miss a session? No problem.
  • Chat, Twitter Integration, SKYPE integration for Q&A
  • Experience-based learning – find out what you need to know from people that are using the technology every day
  • These are NOT sales presentations.
  • 6-Month SSWUG.ORG Membership (or membership extension), included!
  • ALL-ACCESS Pass: SQL Server and Business Intelligence with related SharePoint and .NET technologies – all included, one low price
  • Great vendor hall – learn about the best tools, technologies, publications and partners out there for your shop

Are you jazzed?  Wait, I didn’t make you laugh.  Good sales pitches always include some laughter.  One more bullet point:

Yeah, that’s right.  And there just might be more humor at the one coming up.  So let’s get signed up.

Check out the Ultimate Virtual Conference from SSWUG


ssccdl_couch-laptop1_250x20
In less than a month SSWUG will be hosting the Ultimate Virtual Conference on October 21-24.  This is an excellent opportunity to get the benefits of heading to a conference full of Rock Star speakers while be able to view the content from the comforts of home or your desk.

Reasons to Attend

Some of the details of the event are:

  • More than 75 technical sessions
  • New! Live sessions track – all live sessions, with speaker Q&A/interviews at the end of the day
  • On-Demand access to sessions for 45 days – miss a session? No problem.
  • Chat, Twitter Integration, SKYPE integration for Q&A
  • Experience-based learning – find out what you need to know from people that are using the technology every day
  • These are NOT sales presentations.
  • 6-Month SSWUG.ORG Membership (or membership extension), included!
  • ALL-ACCESS Pass: SQL Server and Business Intelligence with related SharePoint and .NET technologies – all included, one low price
  • Great vendor hall – learn about the best tools, technologies, publications and partners out there for your shop

If that doesn’t jazz you up, then maybe know that one of the best SQL Server humor threads started at the last SSWUG Virtual Conference.

StrateSQL Sessions

I’ll be presenting four sessions at the SSWUG Ultimate Virtual Conference.  My sessions will be:

Solving Business Pains with SQL Server Integration Services

SQL Server Integration Services (SSIS) offers a wide range of features and functionality that can be used to solve business pains within Extract, Transform, and Load (ETL) processes. These pains can be caused by the performance of current ETL process, the maintainability of ETL processes or represent a new pain that SSIS features can resolve. This presentation will provide a demonstration of real world scenarios where features of SSIS were used to solve business pains through modification of existing SSIS packages or through the creation of new SSIS packages. The session will also discuss current best practices in SSIS package design.

Necessarily Evils, Building Optimized CRUD Procedures

Every developer loves them and a lot of DBAs hate them. But there are many and valid reasons for creating generic SELECT, INSERT, UPDATE, and DELETE procedures. In this session, we’ll go through designing CRUD procedures that utilize new and existing SQL features to create CRUD procedures that are optimized for performance.

Improving Daily Imports with Partitioned Tables

At some point, everyone struggles with loading data to OLTP systems. The business needs the data loaded so that users can work through the data. But the users can’t afford the downtime to get the data into the production tables. In this session we’ll review this problem and how to use partitioning to alleviate this issue.

Are You Following Your Own Best Practices?

Everyone has their own best practices that they try to follow. But often times we aren’t sure how well we are following our own or industry best practices. In this session we’ll go into monitoring and managing best practices throughout your environment. The session will look at how to create a solution using policy based management and other tools to report on your compliance to best practices.

One More Reason

For those budget conscious in the current economy, I can even offer a discount code to help nudge those that haven’t quite given in to joining the conference.  With these nine letters you can received a $10 discount off your registration… SPJSUVC09

The Final Pitch

I lied about the one more reason, I’ve got to give one more pitch.

It’s not often that conferences give us the ability to attend all of the sessions that we want when we want to.  With the stellar lineup of presenters and the deep content that is flexible to each persons schedule I highly recommend attending this conference.

This Wednesday ITCare Event – Solving Business Pains with SQL Server Integration Services


search-engine-img

If you’ve been looking for some free training on SQL Server Integration Services, I’ve got the answer for you.  The details are below and you can register for the event here.

Course Overview

ITCare, delivered and hosted quarterly by our Partners is designed to keep you up to date on the cutting-edge technologies. ITCare is a FREE half-day technical seminar series created by IT Professionals for IT Professionals! We’ll address the needs and issues today and show you how to make a real difference in the way that you work.

SQL Server Integration Services (SSIS) offers a wide range of features and functionality that can be used to solve business pains within Extract, Transform, and Load (ETL) processes. These pains can be caused by the performance of current ETL process, the maintainability of ETL processes or represent a new pain that SSIS features can resolve. This presentation will provide a demonstration of real world scenarios where features of SSIS were used to solve business pains through modification of existing SSIS packages or through the creation of new SSIS packages. The session will also discuss current best practices in SSIS package design.

Objectives

1. Understand flexibility that SSIS features provide
2. Demonstrate application of SSIS functionality to business needs
3. Discuss current best practices in SSIS package design

Audience

IT Administrators, IT Managers, Business Intelligence, DBA, SQL Server administrators.

Prerequisites

Understanding of basic ETL concepts and basic T-SQL coding

Course Length

1 Half Day

Speaker: Jason Strate of Digineer, Inc.
Jason Strate has been a Database Architect for over ten years. His experiences include designing and implementing both OLAP and OLTP solutions involving clustering servers, database monitoring and tuning, analysis services, DTS/Integration Services, Reporting Services, and Notification Services. Jason is actively involved with the local PASS chapter (SQL Server User Group); he is currently serving on the Executive Board and has presented at monthly meetings. Jason contributed to Microsoft’s published white paper "Empowering Enterprise Solutions with SQL Server 2005 Enterprise Edition and actively blogs about SQL Server and related technologies. Also this past year, Jason participated in the development of Microsoft Certification exams for SQL Server 2008 and presented at the SSWUG SQL Server Launch Event and the last two SSWUG Virtual SQL Server Conferences.

June ITCare Event – Solving Business Pains with SQL Server Integration Services


search-engine-img
  Some lines got crossed with this month’s New Horizons ITCare event.  The topic and speaker have changed but the date is remaining the same.  Instead of the previous topic, I’ll be presenting on SQL Server Integration Services.  As I mentioned, the date of the event is still June 24th.  The event will be available online and you can register for the event here.

Course Overview

ITCare, delivered and hosted quarterly by our Partners is designed to keep you up to date on the cutting-edge technologies. ITCare is a FREE half-day technical seminar series created by IT Professionals for IT Professionals! We’ll address the needs and issues today and show you how to make a real difference in the way that you work.

SQL Server Integration Services (SSIS) offers a wide range of features and functionality that can be used to solve business pains within Extract, Transform, and Load (ETL) processes. These pains can be caused by the performance of current ETL process, the maintainability of ETL processes or represent a new pain that SSIS features can resolve. This presentation will provide a demonstration of real world scenarios where features of SSIS were used to solve business pains through modification of existing SSIS packages or through the creation of new SSIS packages. The session will also discuss current best practices in SSIS package design.

Objectives

1. Understand flexibility that SSIS features provide
2. Demonstrate application of SSIS functionality to business needs
3. Discuss current best practices in SSIS package design

Audience

IT Administrators, IT Managers, Business Intelligence, DBA, SQL Server administrators.

Prerequisites

Understanding of basic ETL concepts and basic T-SQL coding

Course Length

1 Half Day

Speaker: Jason Strate of Digineer, Inc.
Jason Strate has been a Database Architect for over ten years. His experiences include designing and implementing both OLAP and OLTP solutions involving clustering servers, database monitoring and tuning, analysis services, DTS/Integration Services, Reporting Services, and Notification Services. Jason is actively involved with the local PASS chapter (SQL Server User Group); he is currently serving on the Executive Board and has presented at monthly meetings. Jason contributed to Microsoft’s published white paper "Empowering Enterprise Solutions with SQL Server 2005 Enterprise Edition and actively blogs about SQL Server and related technologies. Also this past year, Jason participated in the development of Microsoft Certification exams for SQL Server 2008 and presented at the SSWUG SQL Server Launch Event and the last two SSWUG Virtual SQL Server Conferences.

May PASSMN Meeting Today

It probably won’t be as interesting as Old School, but come out this afternoon if you are in the Twin Cities for the PASSMN meeting you could attend.  It starts at 3:00 PM and we meet at 2:30 PM for snacks and socializing.  And by snacks, I usually mean pizza but no promises.

We’ll be hearing about SSIS – Team Development, Deployment and Configuration & Securing and Troubleshooting Service Broker.

Feel free to contact me via e-mail (jstrate@digineer.com) or twitter (@stratesql) with questions or for more information.

May PASSMN Meeting (05/19/2009)

The topics and speakers for this months PASSMN meeting have been announced…

SSIS – Team Development, Deployment and Configuration & Securing and Troubleshooting Service Broker

May 19, 2009
3:00 PM – 5:15 PM

SSIS – Team Development, Deployment and Configuration
Speaker: Dan English, Magenic

SQL Server Integration Services (SSIS) provides enterprise-class scalability, advanced data-integration architecture, and high-performance processing. Many enterprise environments are developing centralized services and standards to support their SQL Server Integration Services platform. During this session you will learn considerations and solutions for team development and how to leverage the power of package configurations for deploying packages to multiple environments.

Dan English - Dan is a Principal Consultant with Magenic and has been developing with Microsoft technologies for over 12 years and has over 5 years experience with Data Warehousing and Business Intelligence. He has been working with SQL Server since version 6.5 and now with 2008 looking towards the Kilimanjaro release. Dan has screencasts of SQL Server 2008 and PerformancePoint Server on YouTube and Soapbox ( keyword search – Magenic) and is an avid blogger (http://denglishbi.spaces.live.com). Dan is fully certified with MS SQL Server 2005 and 2008 Business Intelligence. He enjoys keeping in contact with the community at large responding to forum postings on the Microsoft forums and SQL Server Central areas. Dan is also part of the PASSMN 2009 Executive Board.

Securing and Troubleshooting Service Broker
Speaker: Eric Strom, RELS

Tired of reading “Hello World” articles about Service Broker? Looking for more information but not finding good resources on securing and troubleshooting Service Broker applications? In this presentation, I will discuss some good security practices and share some lessons I learned while implementing and troubleshooting a medium-sized Service Broker application. Expect to learn about securing and troubleshooting Service Broker. A basic understanding of the Service Broker architecture is helpful.

Eric Strom is a Senior Database Administrator at the RELS Companies and is a member of the PASSMN 2009 Executive Board. He has been a SQL Server DBA since 2001 and specializes in performance tuning. Eric studied database theory at the University of Minnesota to earn a B.S. in Computer Science. He loves exchanging ideas with peers and is always looking for a good discussion.

Troubleshooting Permission Issues with CREDITIALS


I keep hearing this story and similar variations…

”On a dark and stormy night, I developed a SQL Server Integration Services (SSIS) package.  It worked wonderfully on my desktop development environment.  All the files were properly accessed and the data was processed in a magnificent matter.”

”But suddenly I deployed the SSIS package to production environment, or sometimes even the test environment first.  And BAM! the SQL Agent job I created to execute the package would fail.”

”I laughed and cried and checked all of the settings and permissions and everything is identical to the development environment.  I just don’t know what is going wrong.  Help me Obi-Won, you’re my only hope.”

The first thing I do any time I hear this story is to tell the developer that I think it’s really weird to actually say “open parenthesis” and “close parenthesis”.  But, right after that I check the execution log for the package.  The execution log will usually have all this verbiage around it not being able to this, that, or the other thing with some external resource that the SSIS package is referencing.

But the developer had stated that everything is identical to the development environment but this, of course, doesn’t mean that the SQL Agent account is actually the same between the two environments.  And this is where the problem often lies.  In the development environment the package was executed by a power user, the developer, that could access anything the developer wanted.  In nice and secure SQL environments, the SQL Agent service account will have minimal permissions within SQL Server and no permissions outside of the server that it doesn’t require.

New package.  New requirements.  New network permissions.  New proxy.

As I mentioned, the account executing the SSIS package has different between the environments.  What if I could run the SSIS package on the production environment with the SAME permissions I ran the package with in the development environment.  SQL Server 2005 and 2008 allows for this through the use of CREDENTIALS.  Books Online defines CREDENTIALS as such:

A credential is a record that contains the authentication information (credentials) required to connect to a resource outside SQL Server. This information is used internally by SQL Server. Most credentials contain a Windows user name and password.

The information stored in a credential enables a user who has connected to SQL Server by way of SQL Server Authentication to access resources outside the server instance. When the external resource is Windows, the user is authenticated as the Windows user specified in the credential. A single credential can be mapped to multiple SQL Server logins. However, a SQL Server login can be mapped to only one credential.

To execute a SQL Agent job under the developers account the following four things need to be done:

1. Create the CREDENTIAL using CREATE CREDENTIAL.

-- Create a credential with the account Domain\User and its password

USE [master]

GO

CREATE CREDENTIAL MyCredential WITH IDENTITY = N'Domain\User', SECRET = N'Password'

GO

2. Create a proxy that references the CREDENTIAL using msdb.dbo.sp_add_proxy.

-- Create a proxy and assign the credential to it

USE [msdb]

GO

EXEC msdb.dbo.sp_add_proxy @proxy_name=N'MySSISProxy',@credential_name=N'MyCredential',@enabled=1

GO

3. Grant the proxy access to the SSIS subsystem using msdb.dbo.sp_grant_proxy_to_subsystem.

-- Grant proxy access to the 'SSIS package execution' subsystem, aka #11

USE [msdb]

GO

EXEC msdb.dbo.sp_grant_proxy_to_subsystem @proxy_name=N'MySSISProxy', @subsystem_id=11

GO

4. Assign the Run As value for the SQL Agent Job Step to the proxy.

image

Often by going through these steps I’m able to show that the package that runs in the developer’s development environment under his login will also run in the production environment under his login.   Now identical really means identical and through either adding permissions to the SQL Agent service account or, even better, creating a CREDENTIAL with the appropriate permissions that package can be executed as intended.

Nice and Secure Environments

I should note that using the developer’s login is a temporary troubleshooting technique.  The developer’s login should not be used on an ongoing basis for the SQL Agent job.  This will open up a whole host of issues such as password resets and proxy accounts that stop working with developers logins are disabled.

And as I mentioned above, some people like nice and secure SQL Agent environments.  I am a big fan of those as well.  CREDENTIALS are a great tool that should be used to keep SQL Agent secure.  By limiting the scope of the SQL Agent service accounts, SQL Agent can be prevented from accessing resources and data that it shouldn’t normally access.  I’m not going to get too much into security best practices around this other than to say if restricting SQL Agent permissions isn’t a current consideration you should read the SQL Server 2005 Security Best Practices – Operational and Administrative Tasks white paper.

Tagged

I’ve been tagged…  Jason Massie tagged me this week and I’ll be responding to it in Monday’s blog.

SQL Server 2008 Integration Services Presentation

I mentioned in a previous post that I’d delivered a few presentations on SQL Server 2008.  Nothing really more interesting to say about that today except that the SSIS presentation that I did is attached to this posting.

If there are any questions on any of the content, feel free to ask and I’ll respond either in a comment or in another blog post.