Using python to write specific lines from one file to another file

200 short text files (50kb) that all have a similar format. I want to find a line in each of those files that contains a certain string and then write that line plus the next three lines (but not rest of the lines in the file) to another text file. I am trying to teach myself python in order to do this and have written a very simple and crude little script to try this out. I am using version 2.6.5, and running the script from Mac terminal:

This more or less works and prints the output to the screen. But I would like to print the lines to a new file instead, so I tried something like this:

However, nothing is written to the file. I also tried

and can’t get that to work, either. If anyone can explain what I’m doing wrong or offer some suggestions about what I should try instead I would be really grateful. Also, if you have any suggestions for making the search better I would appreciate those as well. I have been using a test file where the string I want to find is the only text on the line, but in my real files the string that I need is still at the beginning of the line but followed by a bunch of other text, so I think the way I have things set up now won’t really work, either.

Thanks, and sorry if this is a super basic question!

5 Answers 5

As pointed out by @ajon, I don’t think there’s anything fundamentally wrong with your code except the indentation. With the indentation fixed it works for me. However there’s a couple opportunities for improvement.

1) In Python, the standard way of iterating over things is by using a for loop. When using a for loop, you don’t need to define loop counter variables and keep track of them yourself in order to iterate over things. Instead, you write something like this

to iterate over all the items in a list of strings and print them.

2) In most cases this is what your for loops will look like. However, there’s situations where you actually do want to keep track of the loop count. Your case is such a situation, because you not only need that one line but also the next three, and therefore need to use the counter for indexing ( lst[i] ). For that there’s enumerate() , which will return a list of items and their index over which you then can loop.

If you were to manually keep track of the loop counter as in your example, there’s two things:

3) That i = i+1 should be moved out of the if and else blocks. You’re doing it in both cases, so put it after the if/else . In your case the else block then doesn’t do anything any more, and can be eliminated:

4) Now, this will cause an IndexError with files shorter than 500 lines. Instead of hard coding a loop count of 500, you should use the actual length of the sequence you’re iterating over. len(lines) will give you that length. But instead of using a while loop, use a for loop and range(len(lst)) to iterate over a list of the range from zero to len(lst) — 1 .

5) open() can be used as a context manager that takes care of closing files for you. context managers are a rather advanced concept but are pretty simple to use if they’re already provided for you. By doing something like this

the file will be opened and accessible to you as f inside that with block. After you leave the block the file will be automatically closed, so you can’t end up forgetting to close the file.

In your case you’re opening two files. This can be done by just using two with statements and nest them

or, in Python 2.7 / Python 3.x, by nesting two context manager in a single with statement:

6) Depending on the operating system the file was created on, line endings are different. On UNIX-like platforms it’s \n , Macs before OS X used \r , and Windows uses \r\n . So that Lines[i] == searchquery will not match for Mac or Windows line endings. file.readline() can deal with all three, but because it keeps whatever line endings were there at the end of the line, the comparison will fail. This is solved by using str.strip() , which will strip the string of all whitespace at the beginning and the end, and compare a search pattern without the line ending to that:

(Reading the file using file.read() and using str.splitlines() would be another alternative.)

But, since you mentioned your search string actually appears at the beginning of the line, lets do that, by using str.startswith() :

7) The official style guide for Python, PEP8, recommends to use CamelCase for classes, lowercase_underscore for pretty much everything else (variables, functions, attributes, methods, modules, packages). So instead of Lines use lines . This is definitely a minor point compared to the others, but still worth getting right early on.

So, considering all those things I would write your code like this:

As @TomK pointed out, all this code assumes that if your search string matches, there’s at least two lines following it. If you can’t rely on that assumption, dealing with that case by using a try. except block like @poorsod suggested is the right way to go.

Copy lines from one file to another

I trying to create a script that will allow me to designate an input file and take lines from that file and copy them into a separetly designated destination file. So far I have been able to get pretty close to my goal but the issues I am running into and that I cannot seem to find an answer to (that I can understand atleast) is that when I do my search on my string of data if the word «not» or «is» or any other scripting term is in my string it acts like it, meaning if I search on something like «This is not want I want» it takes the not operator into action and wont copy anything. So I guess my question is what can I do to stop this from happening (I’ve added my script to the bottom, be kind its my first attempt =)

My second question maybe asking a little to much out of the program but I’ll ask it anyway. Much like in the above say I have a txt file with 1000 lines of stuff (individual lines not like a paragraph) and 250 of them start with «This is the line I want to copy» but then each say an individual reason why (This is the line I want to copy cause its awesome, cause its purple. etc. etc) as it sits now my script will just pull the first «This is the part I want to copy» and leave it at that, is there a to have it grab the rest of the string from all of the lines?

Again this might be an easy thing but since I am attempting some self education though trial and error some of the terminology I run across in troubleshooting is just as confusing as the problems I am having =)

Copy and paste content from one file to another file in vi

I am working with two files, and I need to copy a few lines from one file and paste into another file. I know how to copy (yy) and paste (p) in the same file. But that doesn’t work for different files. How is this done?

Also, is there a way to cut-paste? I have tried googling, but most of the resources only talk about copy-paste.

18 Answers 18

Since you already know how to cut/yank text, here are a few ideas for pasting it back into another file:

Edit the first file, yanking the text you want. Then open your second file from within vi ( :e /path/to/other/file ) and paste it

Open both files together in a split window and navigate between them using Ctrl + w , Up / Down either by:

• vi -o /path/to/file1 /path/to/file2
• From within the first file, Ctrl + w , s

If you are using Vim on Windows, you can get access to the clipboard (MS copy/paste) using:

» * d d — cut a line (or 3dd to cut three lines)

» * y y — copy a line (or 3yy to copy three lines)

» * p — paste line(s) on line after the cursor

» * P — paste line(s) on line before the cursor

The lets you paste between separate Vim windows or between Vim and PC applications (Notepad, Microsoft Word, etc.).

Use the variations of d like dd to cut.

To write a range of lines to another file you can use:

Where and are numbers (or symbols) that designate a range of lines.

For using the desktop clipboard, take a look at the +g commands.

You can open the other file and type :r file_to_be_copied_from . Or you can buffer. Or go to the first file, go on the line you want to copy, type «qY , go to the file you want to paste and type «qP .

» buffer_name, copies to the buffer. Y is yank and P is put. Hope that helps!

Here’s one way to do it;

• Start Vim and open file1 which is the file you’re working on.
• : e file2 which will bring up file2, the file you want to copy lines from.
• locate the lines you want to copy. If it’s three lines, you hit 3 y y
• : b 1 this will switch to buffer 1, where file1 is
• figure out where you want to insert the lines you yanked, and hit p

You could have both files viewable too. Split the screen with e.g. Ctrl + w s .

As for cutting, d cuts and places the cut stuff in the yank buffer. d d will «cut» a line.

These are all great suggestions, but if you know location of text in another file use sed with ease. :r! sed -n ‘1,10 p’ This will insert 10 lines in an already open file at the current position of the cursor.

I just found that if you add the following line into your vimrc file,

then Vim is using the system clipboard.

I just found the yank way won’t work on the way where I copy contents between different Vim instance windows. (At least, it doesn’t work based on my Vim knowledge. I don’t know if there is another way to enable it to work).

The yank way only works on the way where multiple files are opened in the same window according to my test.

If you want to do that, you’d better use OS cut-copy-past way such as Ctrl + x , Ctrl + c (under Windows).

While editing the file, make marks where you want the start and end to be using

ma — sets the a mark

mb — sets the b mark

Then, to copy that into another file, just use the w command:

These remaps work like a charm for me:

So, when I’m at visual mode, I select the lines I want and press Ctrl + c and then Ctrl + v to insert the text in the receiver file. You could use «*y as well, but I think this is hard to remember sometimes.

This is also useful to copy text from Vim to clipboard.

Goal: save a piece of one file to another file.

Solution:

1. Select the text you want to save:
   • Position the cursor where you want to start selection
   • Press v to select characters OR uppercase V to select whole lines
   • Move the cursor to the end of what you want to select

Save selected text to the new file. Type : w Space and the name of the new file. Actually you’ll see

Then press Enter

Copying text between two buffers (== files) that are opened in the same instance of Vim is no problem:

Simply yank in one buffer with y (assuming you marked a to-copy area in visual mode before), and then paste into the other buffer with p . It also works with different tabs as long as they’re in the same instance of Vim.

How to open two files in the same instance of Vim depends on your system:

• On Win32, there’s an option in the context menu saying Edit with one vim if you select two or more files
• When you’re on the console, you can achieve it with vim file1 file2
• If you use Vim as editor for another tool, be sure to specify the —remote-silent option to ensure that all files are getting opened in the same instance

If you opened the two files in two different instances of Vim, then you have to go with the system clipboard: in the first Vim instance, yank the text into the system clipboard using «+y (again, mark the area to be yanked in visual mode before), then go to the second Vim and paste the clipboard there: «+p .